Skip to content

refactor(provider)!: remove Conn.List and Observe's marker parsing - #247

Merged
gusfcarvalho merged 13 commits into
fx/imp5b-5-serve-state-urlfrom
fx/imp5b-6-provider-no-list
Oct 28, 2026
Merged

gusfcarvalho merged 13 commits into
fx/imp5b-5-serve-state-urlfrom
fx/imp5b-6-provider-no-list

Conversation

@ccf-lisa

@ccf-lisa ccf-lisa Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Task: IMPROVEMENTS item 5(b), stack 5 PR 6 of 8: fx/imp5b-6-provider-no-list on fx/imp5b-5-serve-state-url. Design: P3 (#223).

Workstreams: FND (the internal/provider interface, provider.WriteProblem and internal/provider/fake) and GH (internal/provider/github, its githubtest, and github-provider.md).

Summary

Implements P3 §No rebuild from markers ("What goes" and "Projections stay") and §Rollout PR 6: the provider side of the rebuild's removal, which #246 started.

  • provider.Conn.List is gone: from the interface, the GitHub adapter (github/list.go: the two searches per repo, the scope's repo globs, ListWindow) and the fake (List, MethodList, its glob helper).
    • No other callers. containsFold, validRepo and tokens.NextPage have other users and stay. tokens.Minter.InstallationRepos stays too: MissingInstallations uses it.
    • githubtest loses GET /search/issues (RouteSearch), which only List used.
    • The Provider's scope.orgs bounded only List, so github.Conn no longer keeps it (see Decisions).
  • Observe stops parsing markers:
    • Observation.Projections, projectionsIn and the sentence about them in provider.go are gone.
    • Observe now selects only an issue's comments { totalCount } (a PR's count was never read, so it goes too). With no marker to find, the comment bodies and the ObserveComments follow-up pages had no reader. Observe now reads no untrusted comment text, and makes no follow-up call for comments.
    • The recorded responses under testdata/graphql keep only the count. issue-train.comments-2.json is gone, and the three goldens lose projections.
  • Project keeps reading the App's own markers. markers, markerComment and parsedMarker moved from observe.go to project.go unchanged, so mine and othersOwn behave as before.
  • The fake keeps the projections it was given in its own store (marks, by ref):
    • ProjectionsOn(ref) returns them, as copies sorted by work item; ProjectCall.Changed still compares against them.
    • Re-seeding an object clears them, as replacing its Observation used to.
    • Projections(), the call log RT's and cmd's tests read since feat(serve)!: --state-url, one leader, and no rebuild from markers #246, is unchanged.
    • No RT or cmd file needed a change.
  • The webhook's refusals answer application/problem+json, as receiveWebhook documents (docs(api): readiness without a rebuild, and a store that stopped writing #234). This uses the new provider.WriteProblem(w, status, code, detail) (FND), built on gen.Problem. It writes the status, its text as the title, internal/server's code, the detail (left out on a 5xx, as the server's errorHandler does), and the server's X-Request-Id as instance.
    • The GitHub handler passes server.Code*: 400 invalid-request, 401 unauthenticated, 405 method-not-allowed, 413 payload-too-large, 503 unavailable.
    • The fake's handler (which serve --dev-fake-providers mounts) answers the same way, an oversized body included (413 payload-too-large; it was 400). It can't import internal/server, because the server's package tests import the fake. That is why the writer lives in internal/provider.
    • The codes are unchanged: 202 when stored, 200 for a duplicate, both with no body.
  • Docs:
    • github-provider.md: §Observe and projection has no List, and the markers are for people. The restart gap no longer mentions the rebuild. The handler's problem bodies are described, and there is a new §Decisions taken with P3. 02-architecture.md's and definitions.md's links now point at #observe-and-projection.
    • contracts.md: no List in Conn, no Projections in Observation, no projection round-trip (this supersedes ADR-0001's "every adapter must implement List"), webhook refusals, and the fake's records (ProjectionsOn).

Tests

  • fake_test.go: drops the List cases and the seeded projection. TestProject checks ProjectionsOn (contents, copies, cleared by Seed). TestHandler checks each refusal's problem code and that 202 and 200 have no body. TestConcurrent also calls ProjectionsOn.
  • github/observe_test.go: drops the projection case, and the train issue keeps its own checks. New TestObserveReadsNoComments: an issue with 150 comments is observed with one GraphQL request, its count is 150, and observeQuery selects no body.
  • github/project_test.go:
    • TestProjectRoundTrip replaces TestRebuildRoundTrip: what Project wrote on the issue and the PR parses back, through github.ParseMarker, to the projection.
    • New TestProjectReadsOnlyItsMarkers carries over Observe's golden marker cases for Project. A human's look-alike, the reviewer App's, a deleted account's and a v2 marker own no labels and are never edited. Of two App markers for one work item the oldest is edited. The markers are spread over three pages of comments.
    • TestList and TestListErrors are gone.
  • github/ingress_test.go: each refusal is checked as a problem (content type, status, title, code, the request id as instance, a detail on each 4xx and none on a 5xx).
  • provider/problem_test.go: TestWriteProblem.

Verification

  • make reviewable (with a golangci-lint built with go1.26 on PATH), after merging feat(serve)!: --state-url, one leader, and no rebuild from markers #246's head e680de5, and again at adebedf after merging 536bd87 and the doc fixes: exit 0 both times. That covers the generated-code check, lint with 0 issues, go test -race with every package ok, catalog checks (66 cases ok), and openapi-lint at 100.
  • grep -rnE 'Conn\.List|\.List\(ctx, v1alpha1\.Scope|Observation\.Projections|projectionsIn|o\.Projections|\)\.Projections\b' internal cmd finds nothing.
    • The spec's grep (\.Projections\b) also matches the fake's accessor gh.Projections() / c.Projections(), which is expected, so this one is narrowed to Observation.Projections and projectionsIn.
    • grep -rn 'Conn.List\|\.Projections\b\|projectionsIn' internal cmd | grep -v 'Projections()' is empty too.
  • Review: ccf-review on refactor(provider)!: remove Conn.List and Observe's marker parsing #247 (review 5682090415) found nothing above Nit. Its three Nits are fixed in 546603c: observePerms' comment no longer names List; the PR fragment, the recorded PR responses and githubtest's model drop comments { totalCount }; the fake answers an oversized body 413 payload-too-large (errors.As on *http.MaxBytesError). make reviewable passes at 546603c (exit 0, 0 lint issues).
  • Self-review (lisa, ccf-review rules) took two passes:
    • Pass 1: a CORE-TEST-001 (Observe's comment read untested) and a doc nit, both fixed in 5047b42.
    • Pass 2: doc nits in github-provider.md, fixed in 3a7aee7.
    • CORE-SIZE-001 was settled by the size-exception label (below).
  • No bucket is involved, so nothing here needs a real cloud bucket.
  • One thing only real GitHub can confirm: GraphQL accepts comments { totalCount } without first/last. GitHub only requires them when nodes or edges are selected, but githubtest doesn't check that.

Ownership

Every file is under FND's or GH's Owns line: internal/provider/*.go, internal/provider/fake/, internal/provider/github/ (with githubtest/ and testdata/), github-provider.md (GH's LLD) and contracts.md (FND's LLD). No contract file (api/, internal/apis/, schemas/) changes.

Decisions (Gustavo, 2026-10-28)

  • Size: size-exception applied, with a justification comment on the PR. The PR counts about 1165 lines excluding testdata (+398/−767), mostly the deletion of List, its helpers, githubtest's search and their tests.
  • The Provider's scope.orgs: left as is. It is read by nothing now that List is gone, and it stays in the contract (internal/apis/v1alpha1/config_types.go:78-81, catalog/providers/github-public.yaml:33), unused for now.
  • The problem media type and codes in two places: both kept, each value pinned by tests. That covers provider.ProblemContentType and server.ProblemContentType, the fake's literal codes, and method-not-allowed in server.toProblem and the GitHub handler.
  • observationDigest (internal/controllers/workitem.go, which still deletes a projections key that no longer exists): feat(serve)!: --state-url, one leader, and no rebuild from markers #246 handles it, since it's RT's.
  • Stale List mentions outside the LLDs: fixed here in d3029af, since they are doc lines about the interface this PR changes.
    • design/workstreams/definitions.md:197: the fake seeds objects for Observe, with ProjectionsOn.
    • :384: GH-S1-T03's List is marked as removed with P3.
    • design/02-architecture.md:146: the link now reads §Observe and projection.
    • Both links point at #observe-and-projection, so github-provider.md's explicit observe-list-and-projection anchor is gone, since nothing links to it any more.

Deviations from the spec

  • Observe reads no comments at all, beyond "stop parsing markers". The bodies and the ObserveComments pages had no other reader. Leaving them would keep fetching untrusted text, and up to 10 extra calls per object, for nothing.
  • The fake's webhook handler answers problems too. The spec named only ingress.go, but contracts.md now says every Conn.Handler does, and serve --dev-fake-providers mounts the fake's.
  • The writer is provider.WriteProblem (FND), used by both handlers, rather than a private helper in GH's package. The GitHub adapter can import internal/server's codes, and it does, but the fake can't.
  • githubtest loses its search route and ObserveComments, and github.Conn loses orgs: nothing used them once List and the comment pages were gone.
  • The task line says "PR 6 of 8", as asked when this PR was assigned. 00-common.md's table lists 7.

🤖 Generated with Claude Code

ccf-lisa Bot and others added 4 commits October 28, 2026 07:25
Implements P3 §No rebuild from markers and §Rollout PR 6 (FND + GH):
- provider.Conn loses List; github/list.go, its search helpers and githubtest's search go
- Observation.Projections and the GitHub adapter's projectionsIn go; Observe reads only the
  comment count, so no comment text and no ObserveComments pages
- Project keeps reading the App's own markers; tests move from Observe's goldens to Project
- the fake keeps projections in its own store (ProjectionsOn)
- webhook refusals answer application/problem+json (provider.WriteProblem), as receiveWebhook
  documents; 202 and 200 unchanged
- github-provider.md and contracts.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s, package doc

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e problem writer

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccf-lisa

ccf-lisa Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

size-exception (Gustavo, 2026-10-28): most of this diff is removal: Conn.List with its search and repo-glob helpers, Observe's marker parsing and comment pages, and the test fixtures that fed them (+407/−902). The webhook problem+json change is small and lives in the same adapter.

ccf-lisa Bot and others added 3 commits October 28, 2026 08:39
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erve and projection

Gustavo's decision on #247: fix the provider-interface doc lines here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@gusfcarvalho gusfcarvalho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ccf-review: COMMENT

no findings.

Stack (base-chain): #223 → #239 → #238 → #244 → #236 → #234 → #246 → #247

ccf-lisa Bot and others added 2 commits October 28, 2026 08:49
…, fake's 413

ccf-review on #247 (review 5682090415):
- observePerms' comment no longer names List;
- observeQuery's PullRequest fragment drops comments { totalCount } (nothing reads it), as do
  the recorded PR responses and githubtest's model;
- the fake's handler answers an oversized body 413 payload-too-large, like GitHub's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccf-lisa
ccf-lisa Bot added this pull request to stack #245 October 28, 2026 11:58
ccf-lisa Bot added 4 commits October 28, 2026 09:07
…mp5b-6-provider-no-list

The stack below was rebased onto main; this keeps the branch's own changes on the new base.
@gusfcarvalho
gusfcarvalho merged commit 8ff8b77 into main Oct 28, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant