Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ The publisher reads the build record from GitHub and verifies its workflow, repo

The PR's `rebuild-bundle.yml` must match the default branch before the publisher accepts the build. After changes to that workflow reach the default branch, update existing dependency branches and rerun the build before dispatching the publisher.

Renovate opens a PR when SocketDev publishes a new `sfw-free` release. The custom manager in [`.github/renovate.json`](.github/renovate.json) updates the shared `SFW_VERSION` pin in `src/ci/install-sfw.ts` for all three integrations.
Renovate opens a PR when SocketDev publishes a new `sfw-free` release. The custom manager in [`.github/renovate.json`](.github/renovate.json) updates the shared `SFW_VERSION` pin in `src/ci/install-sfw.ts` for all three integrations. Add the matching SHA-256 digests from the upstream release assets to `src/ci/sfw-release-digests.ts` in that PR. Remove the entry for the previous release.

## Releasing

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -689,7 +689,7 @@ test:

GitLab caching and outputs require `v1.21.0` or newer. Use the same version for the remote template and `setup-ref`.

Extend `.setup-vp-cached` to restore and save dependency data and the pinned `sfw` binary through GitLab's native cache. The runtime copies restored data from `.setup-vp-cache/` into the directory reported by `vp pm cache dir`. It saves a snapshot after setup and again in `after_script`, so packages added by later job scripts are included.
Extend `.setup-vp-cached` to restore and save dependency data and the pinned `sfw` binary through GitLab's native cache. The runtime checks the SFW binary against its pinned SHA-256 digest and runs it from a private temporary directory. It replaces an invalid cached binary with a verified download. The runtime copies restored data from `.setup-vp-cache/` into the directory reported by `vp pm cache dir`. It saves a snapshot after setup and again in `after_script`, so packages added by later job scripts are included.

Set the include input `cache-policy: pull` to restore without uploading a cache. GitLab restores before setup, so this policy must be set in YAML, not changed during a script. Cache snapshots separate OS, architecture, and package manager; the exact lock-file hash determines `SETUP_VP_CACHE_HIT`. A compatible snapshot from an older lock file can still supply packages on a miss.

Expand Down Expand Up @@ -808,7 +808,7 @@ Without `registryUrl`, the runtime preserves existing project `.npmrc` auth entr

- The template supports Microsoft-hosted Linux, macOS, and Windows agents.
- `Cache@2` restores before `vp install` and saves automatically in a post-job step. It has no supported restore-only input, so Azure does not expose GitHub's `cache-save` control. [Azure pipeline caching](https://learn.microsoft.com/en-us/azure/devops/pipelines/release/caching?view=azure-devops)
- When `sfw` is enabled for an install, a separate native cache reuses its version/platform-specific binary. Both portable runtimes support full YAML install entries and retry the known `sfw` command-lookup failure once.
- When `sfw` is enabled for an install, a separate native cache reuses its version/platform-specific binary after a SHA-256 check. Both portable runtimes support full YAML install entries and retry the known `sfw` command-lookup failure once.
- Missing lock files or cache paths degrade to a warning and `SETUP_VP_CACHE_READY=false` instead of failing setup.
- For Azure Artifacts feeds, compose with Azure's `npmAuthenticate` task and/or pass `registryUrl` plus `NODE_AUTH_TOKEN`.

Expand Down
58 changes: 29 additions & 29 deletions dist/azure/index.mjs

Large diffs are not rendered by default.

62 changes: 31 additions & 31 deletions dist/gitlab/index.mjs

Large diffs are not rendered by default.

99 changes: 82 additions & 17 deletions src/ci/install-sfw.test.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,28 @@
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
import { createHash } from "node:crypto";
import {
mkdirSync,
mkdtempSync,
readdirSync,
readFileSync,
rmSync,
symlinkSync,
writeFileSync,
} from "node:fs";
import { realpath } from "node:fs/promises";
import { tmpdir } from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vite-plus/test";
import { setupSfw, SFW_VERSION } from "./install-sfw.js";
import { commandPath } from "./process.js";

vi.mock("./process.js", () => ({ commandPath: vi.fn() }));
vi.mock("./sfw-release-digests.js", () => ({
getSfwAssetDigest: () => createHash("sha256").update("binary").digest("hex"),
}));
const directories: string[] = [];
afterEach(() => {
vi.resetAllMocks();
vi.unstubAllEnvs();
for (const dir of directories.splice(0)) rmSync(dir, { recursive: true, force: true });
});

Expand Down Expand Up @@ -67,28 +81,79 @@ describe("portable sfw preview handling", () => {
});

describe("portable sfw cache", () => {
it("reuses a complete versioned asset without a second download", async () => {
const cacheDirectory = mkdtempSync(path.join(tmpdir(), "setup-vp-sfw-cache-"));
directories.push(cacheDirectory);
const download = vi.fn(async (_url: string, file: string) => {
writeFileSync(file, "binary");
});
const options = {
sfwEnabled: true,
platform: "linux" as const,
arch: "x64",
isMusl: false,
cacheDirectory,
async function fixture() {
const root = await realpath(mkdtempSync(path.join(tmpdir(), "setup-vp-sfw-cache-")));
directories.push(root);
// Keep private staging and the cache separate and clean both after each test.
vi.stubEnv(process.platform === "win32" ? "TEMP" : "TMPDIR", root);
const cacheDirectory = path.join(root, "cache");
const bin = path.join(cacheDirectory, SFW_VERSION, "sfw-free-linux-x86_64", "sfw");
const download = vi.fn(async (_url: string, file: string) => writeFileSync(file, "binary"));
const logWarning = vi.fn();
return {
bin,
download,
logWarning,
options: {
sfwEnabled: true,
platform: "linux" as const,
arch: "x64",
isMusl: false,
cacheDirectory,
download,
logWarning,
},
};
}

it("uses a verified private copy of a cached binary", async () => {
const { bin, download, options } = await fixture();
await setupSfw([{}], { ...options, env: {} });
const env = { PATH: "/bin" };
await setupSfw([{}], { ...options, env });
const command = await setupSfw([{}], { ...options, env });
expect(download).toHaveBeenCalledTimes(1);
const bin = path.join(cacheDirectory, SFW_VERSION, "sfw-free-linux-x86_64", "sfw");
expect(readFileSync(bin, "utf8")).toBe("binary");
expect(env.PATH).toBe(path.dirname(bin) + ":/bin");
expect(command.sfw).toBe(true);
expect(command.executable).not.toBe(bin);
expect(readFileSync(command.executable, "utf8")).toBe("binary");
expect(env.PATH).toBe(path.dirname(command.executable) + ":/bin");
await setupSfw([{}], { ...options, isMusl: true, env: {} });
expect(download).toHaveBeenCalledTimes(2);
});

it("replaces a cache entry with a different digest", async () => {
const { bin, download, logWarning, options } = await fixture();
await setupSfw([{}], { ...options, env: {} });
writeFileSync(bin, "outdated binary");

const command = await setupSfw([{}], { ...options, env: {} });

expect(logWarning).toHaveBeenCalledWith(expect.stringContaining("invalid cached sfw"));
expect(download).toHaveBeenCalledTimes(2);
expect(readFileSync(command.executable, "utf8")).toBe("binary");
expect(readFileSync(bin, "utf8")).toBe("binary");
});

it("skips cache writes through a linked directory", async () => {
const { logWarning, options } = await fixture();
const target = path.join(path.dirname(options.cacheDirectory), "linked-directory-target");
mkdirSync(target);
symlinkSync(target, options.cacheDirectory, process.platform === "win32" ? "junction" : "dir");

const command = await setupSfw([{}], { ...options, env: {} });

expect(readFileSync(command.executable, "utf8")).toBe("binary");
expect(logWarning).toHaveBeenCalledWith(expect.stringContaining("cache path contains a link"));
expect(readdirSync(target)).toEqual([]);
});

it("rejects downloads that do not match the release digest", async () => {
const { bin, download, options } = await fixture();
download.mockImplementation(async (_url, file) => writeFileSync(file, "incomplete download"));
const env = { PATH: "/bin" };

await expect(setupSfw([{}], { ...options, env })).rejects.toThrow("SHA-256 mismatch");
expect(download).toHaveBeenCalledTimes(2);
expect(env.PATH).toBe("/bin");
expect(() => readFileSync(bin)).toThrow();
});
});
187 changes: 136 additions & 51 deletions src/ci/install-sfw.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
import { createWriteStream, existsSync } from "node:fs";
import { chmod, mkdir, mkdtemp, rename, rm, stat } from "node:fs/promises";
import { createHash } from "node:crypto";
import { createReadStream, createWriteStream, existsSync } from "node:fs";
import { chmod, copyFile, lstat, mkdir, mkdtemp, rename, rm } from "node:fs/promises";
import { spawnSync } from "node:child_process";
import type { get as httpGet } from "node:http";
import { tmpdir } from "node:os";
import { setTimeout as sleep } from "node:timers/promises";
import path from "node:path";
import { isWindows } from "./platform.js";
import { commandPath } from "./process.js";
import { resolveSfwEnabled } from "./sfw.js";
import { getSfwAssetDigest, type SfwAsset } from "./sfw-release-digests.js";
import type { ExportVariable, InstallCommand, LogFn, RunInstallEntry } from "./types.js";

// All integrations use this version pin. Renovate updates it through
Expand All @@ -31,7 +34,11 @@ export function isMuslLinux(): boolean {
return existsSync("/etc/alpine-release");
}

export function getSfwAssetName(platform: NodeJS.Platform, arch: string, isMusl: boolean): string {
export function getSfwAssetName(
platform: NodeJS.Platform,
arch: string,
isMusl: boolean,
): SfwAsset {
if (platform === "darwin") {
if (arch === "arm64") return "sfw-free-macos-arm64";
if (arch === "x64") return "sfw-free-macos-x86_64";
Expand Down Expand Up @@ -148,6 +155,118 @@ export function downloadFile(
});
}

async function hasDigest(file: string, expected: string): Promise<boolean> {
const hash = createHash("sha256");
for await (const chunk of createReadStream(file)) hash.update(chunk);
return hash.digest("hex") === expected;
}

async function restoreCachedBinary(
cachedFile: string,
executable: string,
digest: string,
warn: LogFn,
): Promise<boolean> {
try {
if ((await lstat(cachedFile)).isFile()) {
await copyFile(cachedFile, executable);
if (await hasDigest(executable, digest)) {
console.log(`setup-vp: using verified cached sfw ${SFW_VERSION}`);
return true;
}
}
warn(`setup-vp: ignoring invalid cached sfw ${SFW_VERSION}`);
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== "ENOENT") {
warn(`setup-vp: could not use cached sfw ${SFW_VERSION}: ${String(error)}`);
}
}
return false;
}

async function downloadVerifiedBinary(
download: typeof downloadFile,
asset: SfwAsset,
executable: string,
digest: string,
): Promise<void> {
const url = `${SFW_RELEASE_BASE}/${asset}`;
for (let round = 1; ; round += 1) {
try {
await rm(executable, { force: true });
console.log(`setup-vp: installing sfw ${SFW_VERSION} from ${url}`);
await download(url, executable);
if (!(await hasDigest(executable, digest))) {
throw new Error(`SHA-256 mismatch for sfw ${SFW_VERSION} ${asset}`);
}
return;
} catch (error) {
if (round === 2) throw error;
await sleep(2000);
}
}
}

async function prepareCacheDirectory(directory: string): Promise<void> {
// Check ancestors before touching this entry. Windows short names may refer
// to ordinary directories even when their resolved spelling is different.
const parent = path.dirname(directory);
if (parent !== directory) await prepareCacheDirectory(parent);

const entry = await lstat(directory).catch((error: NodeJS.ErrnoException) => {
if (error.code !== "ENOENT") throw error;
return undefined;
});
if (!entry) await mkdir(directory, { recursive: true });
if (!(entry ?? (await lstat(directory))).isDirectory()) {
throw new Error("The sfw cache path contains a link or a non-directory entry.");
}
}

async function saveCachedBinary(file: string, directory: string): Promise<void> {
await prepareCacheDirectory(directory);
const staging = await mkdtemp(path.join(directory, ".setup-vp-"));
try {
const stagedFile = path.join(staging, path.basename(file));
await copyFile(file, stagedFile);
await rename(stagedFile, path.join(directory, path.basename(file)));
} finally {
await rm(staging, { recursive: true, force: true });
}
}

async function prepareVerifiedSfw(
asset: SfwAsset,
name: string,
cacheDirectory: string | undefined,
download: typeof downloadFile,
warn: LogFn,
): Promise<string> {
const digest = getSfwAssetDigest(SFW_VERSION, asset);
if (!digest) throw new Error(`Missing SHA-256 digest for sfw ${SFW_VERSION} ${asset}`);
const cache = cacheDirectory && path.resolve(cacheDirectory, SFW_VERSION, asset);
// Execute only a verified copy in private staging. The cache can live in the checkout.
const sfwDir = await mkdtemp(path.resolve(tmpdir(), "setup-vp-sfw-"));
const sfwBin = path.join(sfwDir, name);

try {
const hit = cache && (await restoreCachedBinary(path.join(cache, name), sfwBin, digest, warn));
if (!hit) {
await downloadVerifiedBinary(download, asset, sfwBin, digest);
if (cache) {
await saveCachedBinary(sfwBin, cache).catch((error: unknown) => {
warn(`setup-vp: could not save verified sfw ${SFW_VERSION}: ${String(error)}`);
});
}
}
await chmod(sfwBin, 0o755);
return sfwBin;
} catch (error) {
await rm(sfwDir, { recursive: true, force: true });
throw error;
}
}

export async function setupSfw(
runInstallEntries: RunInstallEntry[],
options: {
Expand All @@ -169,12 +288,9 @@ export async function setupSfw(
const arch = options.arch ?? process.arch;
const isMusl = options.isMusl ?? isMuslLinux();
const download = options.download ?? downloadFile;
const warn = options.logWarning ?? console.warn;

const effectiveSfw = resolveSfwEnabled(
sfwEnabled,
options.vitePlusVersion ?? "",
options.logWarning ?? console.warn,
);
const effectiveSfw = resolveSfwEnabled(sfwEnabled, options.vitePlusVersion ?? "", warn);
if (!effectiveSfw) return { executable: "vp", sfw: false };

if (runInstallEntries.length === 0) {
Expand All @@ -190,7 +306,7 @@ export async function setupSfw(
return { executable: existing, sfw: true };
}

let asset: string;
let asset: SfwAsset;
try {
asset = getSfwAssetName(platform, arch, isMusl);
} catch {
Expand All @@ -200,46 +316,15 @@ export async function setupSfw(
return { executable: "vp", sfw: false };
}

const cacheDirectory = options.cacheDirectory || env.SETUP_VP_SFW_CACHE_DIR;
const sfwDir = cacheDirectory
? path.resolve(cacheDirectory, SFW_VERSION, asset)
: await mkdtemp(path.resolve(tmpdir(), "setup-vp-sfw-"));
await mkdir(sfwDir, { recursive: true });
const sfwBin = path.join(sfwDir, isWindows(platform) ? "sfw.exe" : "sfw");
const sfwUrl = `${SFW_RELEASE_BASE}/${asset}`;
function activate(): InstallCommand {
const pathSeparator = isWindows(platform) ? ";" : ":";
env.PATH = `${sfwDir}${pathSeparator}${env.PATH || ""}`;
options.exportVariable?.("PATH", env.PATH);
return { executable: sfwBin, sfw: true };
}
if (
await stat(sfwBin).then(
(file) => file.isFile() && file.size > 0,
() => false,
)
) {
console.log(`setup-vp: using cached sfw ${SFW_VERSION}: ${sfwBin}`);
return activate();
}

for (let round = 1; round <= 2; round += 1) {
const downloadDir = await mkdtemp(path.join(sfwDir, ".download-"));
const downloadPath = path.join(downloadDir, "sfw");
try {
console.log(`setup-vp: installing sfw ${SFW_VERSION} from ${sfwUrl}`);
await download(sfwUrl, downloadPath);
if ((await stat(downloadPath)).size === 0) throw new Error("Downloaded sfw binary is empty");
await chmod(downloadPath, 0o755);
await rename(downloadPath, sfwBin);
return activate();
} catch (error) {
if (round === 2) throw error;
await new Promise((resolve) => setTimeout(resolve, 2000));
} finally {
await rm(downloadDir, { recursive: true, force: true });
}
}

throw new Error("failed to install sfw after retrying");
const sfwBin = await prepareVerifiedSfw(
asset,
isWindows(platform) ? "sfw.exe" : "sfw",
options.cacheDirectory || env.SETUP_VP_SFW_CACHE_DIR,
download,
warn,
);
const pathSeparator = isWindows(platform) ? ";" : ":";
env.PATH = `${path.dirname(sfwBin)}${pathSeparator}${env.PATH || ""}`;
options.exportVariable?.("PATH", env.PATH);
return { executable: sfwBin, sfw: true };
}
Loading
Loading