Skip to content

ci: probe the EBADARCH architectures, after the arch-flag hypothesis failed - #25

Closed
Shepdesign wants to merge 5 commits into
ci/codeql-tracer-negative-resultfrom
ci/codeql-native-arch
Closed

Shepdesign wants to merge 5 commits into
ci/codeql-tracer-negative-resultfrom
ci/codeql-native-arch

Conversation

@Shepdesign

@Shepdesign Shepdesign commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

⚠️ Rewritten on 2026-09-25 — the hypothesis this PR opened with was wrong

It shipped ONLY_ACTIVE_ARCH=YES on the theory that an arm64 tracer was failing in an x86_64 compile. Run 36093785096 falsified it. The flag is reverted; what replaces it is a diagnostic, not another guess. The original body predicted this as outcome (c), so at least the prediction was honest.

Based on #24, not main. Retarget when #24 merges.

What the failed experiment established

ONLY_ACTIVE_ARCH=YES worked exactly as specified — the only architecture compiled was arm64, no x86_64 slice anywhere in the log — and Bad CPU type still occurred. So the arch mismatch was not the cause.

The theory is dead on its own terms too. I read the Mach-O fat header of tools/osx64/libtrace.dylib out of the 2.27.1 osx64 bundle:

magic: 0xcafebabe
FAT binary, 2 slices:
   - x86_64
   - arm64

It was never missing a slice for the build. I inferred that mismatch from the …/CodeQL/2.27.1/arm64/… tool-cache path plus a SwiftCompile normal x86_64 line, and never measured it — the same mistake as the 2.27.0 version claim in #24, one step further along.

What it settled, which is the actual win

There is one blocker, not two. #24 recorded the 'self' is immutable errors as possibly independent, cause unestablished. This run named the cause outright:

ViewModifiers.swift:101:21: error: external macro implementation type
'SwiftUIMacros.StateMacro' could not be found for macro 'State()';
compiler plugin '…/usr/bin/swift-plugin-server' could not be loaded:
Bad CPU type in executable

@State is itself a macro, expanded by the same plugin server. When the plugin will not load, @State does not apply, so its nonmutating set never exists — and triggerRefresh.toggle() at :114 and the assignment at :117 become "cannot use mutating member on immutable value". They are downstream of the one failure and need no separate fix. That removes the "second blocker" from the picture entirely.

932b589 universal 6682385 universal 8ad8ad4 arm64-only
Bad CPU type 14 0 2
'self' is immutable 4 4 4
architectures compiled arm64, x86_64 arm64, x86_64 arm64

What this PR now does

Replaces the flag with one continue-on-error step that prints lipo -archs for swift-plugin-server, swift-frontend and $DYLD_INSERT_LIBRARIES. The build command is byte-identical to main's again.

It is an instrument, not a fix. The remaining candidate is an architecture on neither list — arm64e, since dyld refuses an arm64 library inserted into an arm64e process, and Xcode 27 moving helpers to arm64e would explain why this began when the image moved. That is measurable on the runner in one cycle, and I am not going to argue it from here, because arguing it from here has now been wrong twice.

The probe also records in the workflow what is already ruled out, so nobody re-tests it: libtrace carries both slices, and building one architecture does not help.

Demo

The probe's output in the next Analyze (swift) log is the deliverable. Analyze (swift) will still fail — the probe does not fix anything and cannot, by design.

$ python3 -c "import yaml; …"                      # 9 steps now (probe added)
YAML OK, steps: 9
$ git diff origin/main -- .github/workflows/codeql.yml | grep '^[+-]\s*xcodebuild'
                                                   # empty: build command unchanged
$ python3 -m unittest discover -s scripts/tests
Ran 110 tests in 0.040s
OK

Rule check

Non-negotiable 5 — serves its enforcement by trying to make the gate able to run at all. No app behaviour, no endpoint, Sources/ untouched, contents: read unchanged. The probe only reads binary architectures already on the runner; it sends nothing anywhere.

Brand check

None. No UI, no assets, no tokens.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UuHYhiqbmF4dT7sq8a6AYm

The traced build was compiling arm64 + x86_64 while the tracer's
libtrace.dylib is arm64 only. An arm64 library cannot be inserted into the
x86_64 slice's compiler, so dyld refused it and everything that process then
loaded died with EBADARCH — which is why #Preview stopped expanding inside
KeyboardShortcuts and the build exited 65.

Nothing in project.yml or the xcconfigs sets ARCHS, so the Release
configurations take ARCHS_STANDARD. ONLY_ACTIVE_ARCH=YES on the two CodeQL
xcodebuild calls leaves one architecture and nothing the tracer cannot host.

Scoped to this workflow. ci.yml keeps building universal, because that is what
ships and it is untraced, so it never had this problem. CodeQL loses nothing:
the database holds Swift source, and the second architecture only compiled the
same files twice.

This is a hypothesis with a mechanism behind it, not a verified fix, and it
cannot be verified anywhere but on the runner. Two things to watch on the first
green run. The EBADARCH failure is intermittent — zero occurrences in one of the
two runs that motivated this — so one pass does not prove the flag did anything.
And a second, consistent failure sits underneath it: four "'self' is immutable"
errors in KeyboardShortcuts/ViewModifiers.swift, present in both runs and only
under tracing. If those were knock-on from the arch mismatch they go now; if
they were independent they are what turns the job red next, and this changes
nothing about them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuHYhiqbmF4dT7sq8a6AYm
…tead

ONLY_ACTIVE_ARCH=YES did exactly what it claimed — run 36093785096 compiled
pure arm64, no x86_64 slice anywhere — and "Bad CPU type" still occurred. The
flag is reverted rather than left in place looking like a fix.

The theory behind it is dead on its own terms too: libtrace.dylib is a FAT
binary carrying both x86_64 and arm64, read out of the 2.27.1 osx64 bundle's
Mach-O fat header. It was never missing a slice for the build. I inferred that
mismatch from a tool-cache path and a "SwiftCompile normal x86_64" line and
never measured it, which is the same mistake as the 2.27.0 version claim.

In exchange the run settled the other question. There is ONE blocker, not two:
@State is itself a macro, the log names it failing to expand —

  external macro implementation type 'SwiftUIMacros.StateMacro' could not be
  found for macro 'State()'; compiler plugin '…/swift-plugin-server' could not
  be loaded: Bad CPU type in executable

— and the "'self' is immutable" errors are what that looks like downstream.
They need no separate fix and the banner no longer implies they might.

What replaces the flag is a probe, not another guess: one step that prints
lipo -archs for swift-plugin-server, swift-frontend and the inserted dylib. The
remaining candidate is an architecture on neither list, arm64e being the obvious
one, and that is measurable on the runner rather than arguable from here.
continue-on-error, so it can never fail the job.

The build command is byte-identical to main's again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuHYhiqbmF4dT7sq8a6AYm
@Shepdesign Shepdesign changed the title ci: build one architecture under CodeQL so the tracer can be inserted ci: probe the EBADARCH architectures, after the arch-flag hypothesis failed Sep 25, 2026
Shepdesign and others added 3 commits September 25, 2026 04:38
The probe measured the failure instead of inferring it, and it is neither
arm64e nor a missing tracer slice:

  runner (uname -m):    x86_64      <- the job's shell runs under Rosetta
  swift-plugin-server:  arm64       <- arm64 ONLY
  swift-frontend:       arm64       <- arm64 ONLY
  libtrace.dylib:       x86_64 arm64

Xcode 27 dropped the x86_64 slices from its toolchain binaries, while the
Actions runner's shell is still translated x86_64. Untraced this does not
matter — the kernel picks the arm64 slice, which is why ci.yml has been green
throughout. Under the tracer it does: DYLD_INSERT_LIBRARIES makes dyld hold a
child to its parent's architecture so the inserted library matches, and an
arm64-only helper has no x86_64 slice to be held to. Hence

  compiler plugin '.../swift-plugin-server' could not be loaded:
  Bad CPU type in executable

and hence @State never expanding. Running xcodebuild through `arch -arm64`
makes the whole tree arm64, which the arm64-only toolchain and the fat
libtrace both satisfy.

It also explains the intermittency that earlier notes called deterministic and
then called intermittent without explaining: it depends which runner the job
lands on.

Every premise here was read off the runner rather than reasoned about, which is
the difference between this and the two attempts before it. The step prints its
parent architecture so the next reader can confirm the mechanism still holds
rather than trusting this message. The probe stays for the same reason.

Scoped to this workflow; ci.yml is untraced and unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuHYhiqbmF4dT7sq8a6AYm
… worked

The arm64 parent fixed what it targeted. Controlled rather than hopeful: the
probe printed `runner: x86_64` in both runs, so the translated shell was
unchanged and the wrapper was the only difference.

                       c5d7d6a      c2c0166 (arch -arm64)
  Bad CPU type              2                          0
  could not be loaded       2                          0
  'self' is immutable       4                          0

The macros expand, KeyboardShortcuts compiles, and the build reaches this
project's own target for the first time. EBADARCH is settled.

What is red now is a different failure, and in our code rather than a
dependency:

  error: the following command failed with exit code 0 but produced no further
  output
  SwiftCompile normal arm64 (in target 'NetRelish' from project 'NetRelish')

Twice, once per flavour, surrounded by warnings only — no Swift error, no BUILD
FAILED line, no crash, and an exit code of 0 that xcodebuild treats as a failure
regardless. Cause not established, and recorded as not established.

So this adds an instrument rather than a fix: a failure-only step that dumps the
tracer log and the extractor's diagnostics, neither of which reaches the job log
because `analyze` is skipped when the build fails. Measuring first is what turned
the last round from two wrong guesses into a working change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuHYhiqbmF4dT7sq8a6AYm
I reported this as fixed one commit ago. It was not. It was a fail-open, and
the only reason it did not ship as a green CodeQL job with an empty database is
that `database finalize` refuses one:

  Error: CodeQL could not process any code written in Swift.

/usr/bin/arch is SIP-protected and dyld purges DYLD_* across a restricted exec,
so `arch -arm64 xcodebuild` ran with no libtrace inserted. Every symptom I read
as success follows from that:

                       c5d7d6a      arch -arm64 (untraced)
  Bad CPU type              2                           0
  'self' is immutable       4                           0
  build                  fail                     SUCCEEDED
  code extracted            -                        NONE

Three of those four look like a fix. The fourth is explained by nothing except
the tracer being absent, and an untraced build passing is exactly what ci.yml
does on every push — which I had already written down as the reason ci.yml is
green, and then failed to apply to my own result.

The "controlled comparison" I claimed was not one. Holding `runner: x86_64`
fixed while changing the wrapper did not isolate the architecture; it removed
tracing, which changes every signal at once.

So: build command back to byte-identical with main, job back to red on the known
EBADARCH failure rather than green on nothing, and the banner now warns the next
reader off this specific trap instead of recommending it.

The probe grows three lines that measure the purge directly — DYLD_INSERT_LIBRARIES
outside arch, under arch, and re-set past the arch boundary — because whether an
arm64 parent is reachable at all now depends on that, and it is measurable rather
than arguable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UuHYhiqbmF4dT7sq8a6AYm

Copy link
Copy Markdown
Member Author

Correction: I reported this fixed. It was a fail-open.

bc8af8b's message and my report both said arch -arm64 fixed the EBADARCH failure. It did not. It removed the tracer, and 9e15bb4 reverts it.

/usr/bin/arch is SIP-protected, and dyld purges DYLD_* across a restricted exec — so arch -arm64 xcodebuild ran with no libtrace inserted at all. Run 36095774560 is what that looks like from the outside: the build succeeded for the first time all session, and then

Error: CodeQL could not process any code written in Swift.
c5d7d6a arch -arm64
Bad CPU type 2 0
'self' is immutable 4 0
build fail SUCCEEDED
code extracted — NONE

Three of those four read as a fix. The fourth is explained by nothing except the tracer being absent — and an untraced build passing is precisely what ci.yml does on every push. I had already written that down in this very file as the reason ci.yml is green, and then failed to apply it to my own result.

The "controlled comparison" I leaned on was not one. Holding runner: x86_64 fixed while changing the wrapper did not isolate the architecture — it removed tracing, which moves every signal at once. A comparison is only controlled if you can name what it holds constant, and I couldn't have: I never checked that the tracer was still attached.

What nearly shipped

Had database finalize not refused an empty database, this would have been a green CodeQL job that had seen no code. That is the fail-open this entire mechanism exists to prevent, and it would have come from the person maintaining it. CodeQL's own no-source-seen check is the only thing that caught it — not the gate, not the tests, not me.

That is the fourth distinct way this mechanism has been one step from reporting clean on nothing, and the first where I was the one holding the step.

Where this leaves the PR

  • Build command byte-identical to main's again. The job is back to red on the known EBADARCH failure, which is the honest state — red on a real failure beats green on nothing.
  • The banner now warns the next reader off arch specifically, with the four-signal table, instead of recommending it.
  • The probe grows three lines that measure the purge rather than assuming it: DYLD_INSERT_LIBRARIES outside arch, under arch, and re-set past the arch boundary. That last one decides whether an arm64 parent is reachable at all — if the variable survives an explicit re-set, there may be a real version of this fix; if not, the arch route is closed and the remaining path is upstream.

Still true, and not affected by any of this

@State being a macro, so the 'self' is immutable errors are knock-on rather than a second bug — that came from a traced run and stands.

Also worth flagging: the exit code 0 but produced no further output failure I documented in bc8af8b appeared in both runs, but the build step failed in one and succeeded in the other. So it is not the blocker I took it for; it is noise that sometimes fails the step. That was a second thing I got wrong in the same commit, and it is now stated as unexplained rather than as a finding.


Generated by Claude Code

Copy link
Copy Markdown
Member Author

Handoff — one measurement in flight, and how to read it

I'm standing down on this PR. Run 36096284681 (9e15bb4) is still going and carries the one open question. Its logs aren't served until the job finishes, so I could not capture the answer before stopping. It needs no further code from anyone — just reading.

In the Analyze (swift) log, find the probe step and look at three lines:

DYLD outside arch:      <path>          ← expect the libtrace path
DYLD under arch:        ???             ← THE ANSWER
DYLD re-set under arch: ???             ← the fallback
DYLD under arch Meaning What follows
<unset> dyld purged it across the SIP-protected arch exec — confirms why arch -arm64 silently disabled tracing check the third line
a path the purge theory is wrong, and the untraced run has some other cause — re-open from there

And if the second line is <unset> but the third shows a path, an explicit re-set survives the arch boundary and there is a real version of the arm64 fix: arch -arm64 /usr/bin/env DYLD_INSERT_LIBRARIES="$DYLD_INSERT_LIBRARIES" xcodebuild …. If the third is also <unset>, the arch route is closed and nothing in this repo gets Analyze (swift) green — it needs Xcode's helpers to regain x86_64 slices, or the runner to stop translating, or CodeQL's tracer to handle the mismatch.

Verify before trusting any future green here. The lesson of bc8af8b is that this job can pass by seeing nothing. Any change that turns it green must also show a non-empty database — database finalize succeeding and the gate step actually running, not skipped.

State of both PRs

Neither is required for anything on main; the gate and its 110 tests landed in eae0e41 and are untouched by all of this.

What is established, all measured rather than argued

Fact Evidence
swift-plugin-server, swift-frontend are arm64-only lipo -archs on the runner
runner shell is x86_64 (Rosetta) uname -m on the runner
libtrace.dylib is fat (x86_64 + arm64) Mach-O fat header of the 2.27.1 bundle
@State is a macro — 'self' is immutable is knock-on, not a second bug named in a traced run's error
trace = false cannot prevent injection reproduced against 2.27.0
Swift has no build-mode: none swift/codeql-extractor.yml
arch strips the injection — do not reach for it run 36095774560, and the probe above

All of it is in the workflow banner, so nobody has to repeat today.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant