Skip to content

fix(windows): ARM64 installs — nsis7z-safe payloads, per-arch manifests, single draft - #1208

Open
JamieRuderman wants to merge 34 commits into
mainfrom
fix/windows-native-arch-updates
Open

fix(windows): ARM64 installs — nsis7z-safe payloads, per-arch manifests, single draft#1208
JamieRuderman wants to merge 34 commits into
mainfrom
fix/windows-native-arch-updates

Conversation

@JamieRuderman

@JamieRuderman JamieRuderman commented Sep 4, 2026

Copy link
Copy Markdown
Member

Why

3.47.1 failed to install on every Windows ARM64 machine with "The Remote.It agent service could not be installed." Two independent defects, found in sequence:

  1. Manifest ordering. electron-updater ≤ 6.6.2 (every release through 3.46.1) installs the first entry of latest.yml files, written in build-completion order. In 3.47.1 that entry became the 301 MB multi-arch installer. Hot-fixed by hand on v3.47.1 (ia32 first).
  2. The arm64 payload itself is unextractable. electron-builder 26.15's 7-Zip compresses ARM64 executables with the newer ARM64 branch filter; NSIS's nsis7z.dll (built 2019) can't decode it and silently skips those entries. The installer removes the old install, then lands app.asar, every .pak, and the x86 elevate.exe — but no Remote.It.exe, no DLLs, no resources\*.exe. This is the 3.47.1 universal failure, the 3.47.1 standalone arm64 installer on remote.it/download, and the 3.48.1 pre-release on an ARM64 Surface. 3.46.1 (26.8.1) used BCJ2 throughout and works. Defender ruled out on the machine.

What

  • Payload compressionELECTRON_BUILDER_7Z_FILTER=BCJ2 in the build-electron script; reproduces the 26.8.1 method string exactly. electron/scripts/verify-win-installers.js carves each installer's payload by its 7z signature, lists its coders, and fails the build — pulling the installers from the draft — on any coder nsis7z lacks or any missing executable. Validated locally: 3.48.1 arm64 fails on exactly the files that went missing; 3.48.1 ia32 and 3.46.1 arm64 pass.
  • Per-arch installers onlynsis.buildUniversalInstaller: false; names unchanged.
  • Deterministic manifestsfinalize-win-update-manifests.js orders files ia32 → x64 → arm64, verifies sha512/size on disk, writes latest-{ia32,x64,arm64}.yml, re-uploads with --clobber. Uploads only into a draft; a published version fails the job; several drafts sharing the tag fail the job.
  • One draft, created up frontprepare job; runners racing on "release doesn't exist" produced two drafts with the installers split between them.
  • Emulated builds migrate to nativeAutoUpdater sets the GitHub feed's channel option to latest-<arch>. That option only renames the manifest GitHubProvider fetches from the release it picks by its normal rules, so no API call, no pinning, and no hand-rolled channel logic; autoUpdater.channel is never set, because that one also changes which tags are considered and breaks every pre-release user's checks. If the picked release predates the per-arch files the app falls back to latest.yml (current arch). Tags with a semver pre-release id (-beta.1) update on the current arch only, since GitHubProvider asks for beta.yml and falls back to latest.yml itself; the native hop waits for the next plain-version tag.
  • Cross-arch upgrade (found on the Surface at 3.48.2 ia32 → 3.48.3 arm64) — an ia32 install registers in the 32-bit registry view; the x64/arm64 installers read the 64-bit view, so electron-builder skipped the old uninstaller, the old agent kept remoteit.exe locked, extraction couldn't replace it ("Remote.It cannot be closed… Retry"), and the result was an arm64 app with an ia32 agent plus two Programs-and-Features entries. preInit now copies a 32-bit-only UninstallString into the 64-bit view so the normal upgrade path runs the old uninstaller (electron-builder derives the old folder from the uninstaller path); customInstall removes the stale 32-bit keys only after that copy. Only the uninstaller entry is copied — a mirrored InstallLocation is adopted as $INSTDIR, which is how the 3.48.3 → 3.48.4 pre-release hop put the arm64 app under Program Files (x86) (found by the self-review below); a public ia32 → native upgrade lands in Program Files. Every currently-ia32 machine takes this hop when steering moves it native.
  • Installer — checks resources\remoteit.exe exists before agent install (this is what turned "agent service could not be installed" into "remoteit.exe is missing" and made defect 2 diagnosable); distinguishes "could not start" from a non-zero exit.
  • verify-binaries — magic-byte check of every bundled binary per arch.
  • Preferences wipe (pre-existing, found during the live test)ui.ts emits { ...preferences, language } on startup; before the backend's preferences reach the renderer that spread is {version:'', cliVersion:''}, and preferences.set replaced the file with it — clearing autoUpdate (no more update checks) and allowPrerelease. Preferences.set now merges, and ui.ts sends only { language } so the un-hydrated snapshot never reaches the backend at all (the merge alone still persisted version: '' / cliVersion: ''). Seen on the Surface at first start of 3.48.2.
  • Brand-aware release steps (Codex review)electron/scripts/release-repo.js resolves the publish repository from brands/<brand>/config.ts (cachengocachengo/desktop) before npm install; prepare, the guard's asset cleanup and the manifest upload all target it. verify-win-installers.js derives installer and app names from the branded productName (Telepath-Installer-*.exe, Telepath.exe) instead of hard-coding Remote.It.
  • Codex rounds 2–3 and /code-reviewRELEASE_TOKEN is used only when the brand publishes to another repository (a token scoped to cachengo/desktop no longer breaks remoteit releases); the write-access check and the draft run on tag builds only, so branch builds of a brand keep working as a smoke test; a transient gh release view failure no longer creates a second draft; a branch build never touches a draft (no manifest upload, no installer cleanup) and no longer fails when its version is already published; --target $GITHUB_SHA only for same-repository builds. Steering no longer derives <channel>-<arch>.yml from the tag — electron-builder's GitHub publisher only ever writes latest.yml, so that lookup could never succeed on a pre-release tag. js-yaml is declared (it was only reachable through hoisting) and the dead 7zip-bin lookup is gone. Round 4: a retried upgrade after an aborted run recognises the already-mirrored uninstaller entry and still cleans the 32-bit keys; a steered fallback that finds only an older release with the per-arch manifest falls through to the unsteered feed instead of reporting no update.
  • RELEASE.md — payload compression, manifests, prepare job, brand repository and token, hot-fix recipe.
  • Versions 3.48.0 → 3.48.5 via npm version on the branch (pre-release test builds). Tags are annotated and point at their bump commits — re-point at the shipped commit at release time; merge with a merge commit or rebase.

Verified

  • Tag builds with prepare: one draft, 25 assets, no universal installer, hashes match, guard passes (BCJ2 LZMA LZMA2), live v3.47.1 untouched.
  • Public-user path, live on an ARM64 Surface (Windows 11, Surface Pro X): installed 3.47.1 from remote.it/download (ia32) → enabled pre-releases → updated to 3.48.3 ia32 by process.arch match (clean CustomRemoveFiles 3.47.1); the preferences race fired at 3.48.3's first start and the merge fix absorbed it (settings intact) → steered to 3.48.4 arm64: log shows Mirrored 32-bit registration to the 64-bit viewCustomRemoveFiles 3.48.3CustomInstall 3.48.4, no lock dialog, Remote.It.exe and every resources\*.exe arm64, agent running, a single Programs-and-Features entry in the 64-bit view, preferences intact, and the new process reports nativeArch: null.
  • x64 NUC: 3.47.1 → 3.48.0 chose x64 by process.arch, clean install, not steered.
  • updateChannel unit tests and the full electron suite (49) pass; tsc clean.

Exposure

  • remote.it/download's Windows ARM64 link (v3.47.1) was broken from Sept 1 to Sept 10 — a clean install destroyed any existing install. Stopgap applied 2026-09-10: the v3.47.1 Remote.It-Installer-arm64.exe asset was replaced with the ia32 build, latest.yml cut to ia32+x64, the release notes annotated, and the S3 mirror re-run; the public link now serves a working 32-bit build (verified byte-for-byte). Native arm64 for the public still needs a signed 3.48.x promoted to Latest.
  • The v3.48.0 / v3.48.1 pre-releases had their arm64 installer and latest-arm64.yml removed so no pre-release user can be steered into it; 3.48.2 supersedes both.

@aws-amplify-us-west-1

Copy link
Copy Markdown

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-1208.d20k671nqqv4kl.amplifyapp.com

@JamieRuderman JamieRuderman changed the title fix(windows): per-arch installers and deterministic update manifests fix(windows): per-arch installers, deterministic update manifests, single draft release Sep 4, 2026
@JamieRuderman JamieRuderman changed the title fix(windows): per-arch installers, deterministic update manifests, single draft release fix(windows): ARM64 installs — nsis7z-safe payloads, per-arch manifests, single draft Sep 10, 2026
@JamieRuderman

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9acb4ed68d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

}

const distDir = path.resolve(process.argv[2] || 'dist')
const installers = fs.readdirSync(distDir).filter(f => /^Remote\.It-Installer-(ia32|x64|arm64)\.exe$/.test(f))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Derive installer names from the selected brand

When the workflow runs with brand=cachengo, brand-electron.ts changes productName to Telepath, so the configured artifact pattern produces Telepath-Installer-<arch>.exe. This hard-coded Remote.It filter finds no installers and exits with status 1, making every Cachengo Windows release fail verification while leaving its actual assets untouched by the similarly hard-coded cleanup. Derive the expected name from the branded package configuration or match the architecture suffix independently of the product name.

Useful? React with 👍 / 👎.

Comment thread .github/workflows/build-electron.yml Outdated
if gh release view "$VERSION" --repo "$GITHUB_REPOSITORY" --json isDraft >/dev/null 2>&1; then
echo "release $VERSION already exists"
else
gh release create "$VERSION" --repo "$GITHUB_REPOSITORY" --draft --title "${VERSION#v}" --notes "" --target "$GITHUB_SHA"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Create the draft in the branded release repository

When brand=cachengo is dispatched from this repository, branding changes the package repository—and therefore electron-builder's publish destination—to cachengo/desktop, but this creates the coordinating draft in $GITHUB_REPOSITORY (remoteit/desktop). The platform publishers therefore do not share the prepared draft, and the later manifest step also operates on the wrong release. Resolve the owner/repository from the selected brand and use it consistently for all gh operations.

Useful? React with 👍 / 👎.

@JamieRuderman

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 280f393da0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +56 to +58
- name: Resolve the release repository for the brand
id: repo
run: echo "RELEASE_REPO=$(node electron/scripts/release-repo.js)" >> $GITHUB_OUTPUT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Authenticate writes to the branded release repository

When brand=cachengo is dispatched from remoteit/desktop, this resolves RELEASE_REPO to cachengo/desktop, but GH_TOKEN is still ${{ secrets.github_token }} from the workflow repository. GitHub's automatic token is scoped to the repository containing the workflow, so the new draft-creation step receives a permission error before the build matrix starts, and the later asset operations have the same problem. Supply a token or GitHub App credential with write access to the selected repository.

Useful? React with 👍 / 👎.

Comment thread .github/workflows/build-electron.yml Outdated
if gh release view "$VERSION" --repo "$RELEASE_REPO" --json isDraft >/dev/null 2>&1; then
echo "release $VERSION already exists in $RELEASE_REPO"
else
gh release create "$VERSION" --repo "$RELEASE_REPO" --draft --title "${VERSION#v}" --notes "" --target "$GITHUB_SHA"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Create cross-repository tags from a destination commit

On the first cachengo release for a version, the tag normally does not exist in cachengo/desktop, so gh release create must create it there; however, $GITHUB_SHA identifies the commit checked out from the source repository and is not guaranteed to exist in the destination. As gh release create --help defines --target as a target branch or full commit SHA for automatic tag creation, the API rejects this whenever the repositories do not share that exact object. Use a branch or commit that exists in RELEASE_REPO, or pre-create/push the destination tag.

Useful? React with 👍 / 👎.

Comment thread electron/src/AutoUpdater.ts Outdated
Comment on lines +199 to +200
const newest = data.find(item => !item.draft && (autoUpdater.allowPrerelease || !item.prerelease))
return newest?.assets?.some(a => a.name === asset) ? newest.tag_name : undefined

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the current prerelease channel while steering

For an emulated Windows client already on a beta build with prerelease updates enabled, this treats every non-draft prerelease as eligible and pins whichever release the API returns first. If a newer alpha release exists, the beta client is therefore steered to its latest-<arch>.yml, bypassing electron-updater's normal rule that prevents beta users from moving onto alpha. Filter candidates using the current version's prerelease channel semantics before selecting the pinned release.

Useful? React with 👍 / 👎.

@JamieRuderman

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19262a3cfe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/build-electron.yml Outdated
GH_TOKEN: ${{ secrets.github_token }}
# A brand's release publishes to the brand's own repository, which the automatic token
# cannot reach; RELEASE_TOKEN (optional) supplies one that can.
GH_TOKEN: ${{ secrets.RELEASE_TOKEN || secrets.github_token }}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Select the release token only for cross-repository builds

When RELEASE_TOKEN is a least-privilege token granting access only to cachengo/desktop, it still replaces github_token for the default remoteit build. Since the permission check is skipped when RELEASE_REPO == GITHUB_REPOSITORY, the subsequent draft creation fails because that token cannot write to remoteit/desktop, blocking every normal release while the Cachengo credential is configured. Choose the token after resolving the destination repository, or fall back to github_token for same-repository builds.

Useful? React with 👍 / 👎.

Comment thread electron/src/AutoUpdater.ts Outdated
Comment on lines +216 to +217
const channel = releaseChannel(newest.tag_name) || 'latest'
return newest.assets?.some(a => a.name === `${channel}-${nativeArch}.yml`)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the update-manifest channel emitted by the build

For a semantic prerelease such as v3.49.0-beta.2, this derives beta-arm64.yml, but the current GitHub electron-builder configuration has no explicit publish channel and emits latest.yml; the finalizer consequently creates latest-arm64.yml, not beta-arm64.yml. The asset check therefore rejects every such release and emulated clients remain on their old architecture, despite the new beta/alpha eligibility logic. Align the generated per-architecture manifest names with the tag-derived channel or have steering request the channel actually published.

Useful? React with 👍 / 👎.

@JamieRuderman

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 419e5db94b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

echo "::error::$drafts draft releases are tagged $VERSION. Keep one, delete the rest, and re-run - see RELEASE.md."
exit 1
fi
gh release upload "$VERSION" $(cat manifests.txt) --repo "$RELEASE_REPO" --clobber ;;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Gate draft manifest uploads on tag builds

When a branch smoke build runs while a draft for the current package version exists, this draft arm executes even though prepare intentionally skips publishing for branches, and replaces that pending release's manifests with files generated by the branch build. This can make the draft point at different or unsigned installers; gh release upload --help confirms that --clobber deletes existing assets before uploading replacements. Require GITHUB_REF_TYPE=tag before this upload (and before the analogous installer cleanup) so branch builds cannot mutate a release draft.

Useful? React with 👍 / 👎.

@JamieRuderman

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a4ea2700de

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread electron/installer.nsh Outdated
; uninstaller was skipped. Copy only its entry: see RELEASE.md, "Cross-architecture upgrades".
SetRegView 64
ReadRegStr $0 HKLM "${UNINSTALL_REGISTRY_KEY}" UninstallString
${if} $0 == ""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve cleanup state when a prior mirror already exists

If a native installer is canceled or aborts after preInit writes the 64-bit value but before customInstall, that registry change persists. On retry, this guard sees the mirrored UninstallString and never sets Mirrored32; after uninstalling the ia32 app, customInstall therefore skips deleting its 32-bit keys, leaving a ghost Programs and Features entry alongside the native installation. Detect and mark an existing mirror, or defer/roll back the mirror, so retried upgrades still clean the original registry view.

Useful? React with 👍 / 👎.

Comment thread electron/src/AutoUpdater.ts Outdated
Comment on lines 210 to 211
await autoUpdater.checkForUpdatesAndNotify()
this.emitStatus()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fall through when a fallback release offers no update

When the latest selected release lacks latest-<arch>.yml but an older release still contains it, this generic check succeeds even if that manifest's version is already at or below the installed version. Returning true then prevents the outer recovery path from calling checkWithoutSteering, so the client never considers the newer release's usable latest.yml and remains stuck. Inspect the check result and continue with the unsteered feed when the pinned manifest offers no update.

Useful? React with 👍 / 👎.

…rough when the fallback release is not newer
@JamieRuderman

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: cbf36010a7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant