From bda8971f2c5362a9233ed0eaaa1304fd1a1c1011 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Thu, 5 Nov 2026 19:15:58 +0100 Subject: [PATCH 01/19] test(e2e): run the pod tests against CSS 8 with Linked Web Storage The Linked Web Storage protocol is being written now, and the Solid CG and the LWS WG asked applications to test against servers that implement it. The Community Solid Server's LWS work isn't released: it lives on a branch of jeswr's fork (pull request 126 there), on the same 8.0.0-alpha.3 as css-8. css-lws builds that branch at one commit and runs it as an advisory server, so its results are reported, not a gate. Its configuration is the fork's solid-lws.json, the Solid Protocol and LWS side by side, in memory and with the root as the storage, as the tests need. Every test passes there with both Solid clients, as on css-8. The fork's LWS-only configuration isn't in the matrix: it serves a Solid-OIDC request as anonymous, so the app can't sign in there, and the tests send no credentials, so they can't show it. Refs: #594 --- .github/workflows/ci.yml | 2 +- .github/workflows/interop.yml | 2 +- cspell.config.yaml | 1 + docs/data-model.md | 2 +- docs/testing.md | 20 +++++++++++++ e2e/pod/servers.test.ts | 3 ++ e2e/pod/servers.ts | 11 +++++++ e2e/pod/servers/css-lws/.dockerignore | 5 ++++ e2e/pod/servers/css-lws/Dockerfile | 35 ++++++++++++++++++++++ e2e/pod/servers/css-lws/compose.yml | 21 +++++++++++++ e2e/pod/servers/css-lws/css-lws-e2e.json | 38 ++++++++++++++++++++++++ 11 files changed, 137 insertions(+), 3 deletions(-) create mode 100644 e2e/pod/servers/css-lws/.dockerignore create mode 100644 e2e/pod/servers/css-lws/Dockerfile create mode 100644 e2e/pod/servers/css-lws/compose.yml create mode 100644 e2e/pod/servers/css-lws/css-lws-e2e.json diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 50e124e35..0e13a66a6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -578,7 +578,7 @@ jobs: # the run; on main, each runs to its end, saving its images. fail-fast: ${{ github.ref != 'refs/heads/main' }} matrix: - server: [css-8, pivot, nextcloud, jss] + server: [css-8, pivot, nextcloud, jss, css-lws] steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: diff --git a/.github/workflows/interop.yml b/.github/workflows/interop.yml index 52b88a22f..4ac273d09 100644 --- a/.github/workflows/interop.yml +++ b/.github/workflows/interop.yml @@ -56,7 +56,7 @@ jobs: strategy: fail-fast: false matrix: - server: [css-8, pivot, nextcloud, jss] + server: [css-8, pivot, nextcloud, jss, css-lws] steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: diff --git a/cspell.config.yaml b/cspell.config.yaml index f100c80fc..ce920ae08 100644 --- a/cspell.config.yaml +++ b/cspell.config.yaml @@ -161,6 +161,7 @@ words: - Gitea - igrant - Inrupt + - jeswr - ixuz - katex - linkify diff --git a/docs/data-model.md b/docs/data-model.md index dfebe567e..bef73d4c0 100644 --- a/docs/data-model.md +++ b/docs/data-model.md @@ -221,7 +221,7 @@ flowchart LR Every blocking server the end-to-end tests start ([testing.md](testing.md#commands)), and Pivot and Community Solid - Server 8, keep another app's file and its folder, and delete a folder + Server 8 (with LWS too), keep another app's file and its folder, and delete a folder holding only Solid Memo's documents with its access rules ([foreignData.integration.test.ts](../e2e/pod/src/foreignData.integration.test.ts)). Solid-Nextcloud lists a folder's ACL among its contents, so there such diff --git a/docs/testing.md b/docs/testing.md index 3df5bab10..a289524c7 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -74,6 +74,7 @@ or on Ctrl-C. The test names say which server and version: | `pivot` | Pivot, the server of solidcommunity.net (the Community Solid Server 7 with Pivot's components), in memory | built here from its lockfile ([servers/pivot/](../e2e/pod/servers/pivot/Dockerfile)) with a configuration of its own, its root open | advisory | | `nextcloud` | Solid-Nextcloud: Nextcloud 32 with the Solid app, on SQLite, the pod `apps/solid/~alice/storage/` | built here from a commit of the app ([servers/nextcloud/](../e2e/pod/servers/nextcloud/Dockerfile)), installed on every start, alice's pod opened by a hook | advisory | | `jss` | JavaScript Solid Server, files in the container, open to anyone (`--public`, which skips its access control) | built here from its lockfile ([servers/jss/](../e2e/pod/servers/jss/Dockerfile)), unmodified and pushed nowhere (it's AGPL) | advisory | +| `css-lws` | Community Solid Server 8 with Linked Web Storage (LWS): the Solid Protocol and LWS side by side, in memory | built here from a commit of a fork not yet merged ([servers/css-lws/](../e2e/pod/servers/css-lws/Dockerfile)), with a configuration of its own, its root open | advisory | A **blocking** server's tests gate CI, and so deploys and Renovate's merges: every test passes, or skips for a reason it gives. An @@ -101,6 +102,25 @@ in a run the test files run one at a time (`serialFiles` in `servers.ts`). css-8 becomes blocking at 8.0.0, and css-6 then goes. +css-lws tests the [LWS](https://www.w3.org/TR/lws10-core/) work in the +Community Solid Server before any release has it: the branch +`feat/lws-next-major` of +[jeswr/CommunitySolidServer](https://github.com/jeswr/CommunitySolidServer/pull/126), +on the same 8.0.0-alpha.3 as css-8, so a failure only css-lws shows +comes from the LWS changes. Its configuration is the fork's +`config/solid-lws.json` in memory, with the root as the storage. The +fork's LWS-only configuration isn't tested: it treats the app's +Solid-OIDC requests as anonymous, so the app can't sign in there. The +image is built from code nobody has reviewed, on CI's runners and on the +machine of anyone who runs `SOLID_SERVERS=all`. Its dependencies' +install scripts don't run, and its final stage keeps only what the +server runs. GitHub's dependency review doesn't see the lockfile it +installs from. A person approves each newer commit on Renovate's +Dependency Dashboard before Renovate proposes it. css-lws goes once a +Community Solid Server release has LWS, or when its commit can't be +fetched any more: a commit that can't be fetched fails the `contract` +job of every pull request. + Each server is a compose file, `e2e/pod/servers//compose.yml`, asking only for `E2E_PORT` (the port on 127.0.0.1 the harness picked, which the server must be told) and `E2E_SECRET` (a password made up for the run). It publishes on diff --git a/e2e/pod/servers.test.ts b/e2e/pod/servers.test.ts index 7beacdbb3..ffef4d1d8 100644 --- a/e2e/pod/servers.test.ts +++ b/e2e/pod/servers.test.ts @@ -231,6 +231,9 @@ describe("the names in the tests", () => { pivot: expect.stringMatching(/^Pivot \d+\.\d+\.\d+ \(Community Solid Server 7\.\d+\.\d+\)$/), nextcloud: expect.stringMatching(/^Solid-Nextcloud [0-9a-f]{8} \(Nextcloud 32\.\d+\.\d+\)$/), jss: expect.stringMatching(/^JavaScript Solid Server \d+\.\d+\.\d+$/), + "css-lws": expect.stringMatching( + /^Community Solid Server with LWS 8\.\d+\.\d+(-[\w.]+)? \([0-9a-f]{8}\)$/, + ), }); }); }); diff --git a/e2e/pod/servers.ts b/e2e/pod/servers.ts index 473a3f8e6..6fd95daa0 100644 --- a/e2e/pod/servers.ts +++ b/e2e/pod/servers.ts @@ -177,6 +177,17 @@ export const SERVERS = { urlFromHost: true, signedIn: { login: "jss", host: "127.0.0.1" }, }, + "css-lws": { + label: "Community Solid Server with LWS", + tier: "advisory", + internalPort: 3000, + podPath: "", + readyPath: "", + startTimeoutMs: 60_000, + // The fork's commit, on the release css-8 runs, so a failure only it shows comes from its LWS changes. + version: () => + `${dockerfileArg("css-lws", "CSS_LWS_VERSION")} (${dockerfileArg("css-lws", "CSS_LWS_SHA").slice(0, 8)})`, + }, } satisfies Record; export type ServerId = keyof typeof SERVERS; diff --git a/e2e/pod/servers/css-lws/.dockerignore b/e2e/pod/servers/css-lws/.dockerignore new file mode 100644 index 000000000..d88f1735e --- /dev/null +++ b/e2e/pod/servers/css-lws/.dockerignore @@ -0,0 +1,5 @@ +# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +# +# SPDX-License-Identifier: MIT + +expected-failures.json diff --git a/e2e/pod/servers/css-lws/Dockerfile b/e2e/pod/servers/css-lws/Dockerfile new file mode 100644 index 000000000..0c9ae9361 --- /dev/null +++ b/e2e/pod/servers/css-lws/Dockerfile @@ -0,0 +1,35 @@ +# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +# +# SPDX-License-Identifier: MIT + +# The Community Solid Server 8 with Linked Web Storage (LWS) for the +# end-to-end tests (docs/testing.md): the branch of jeswr's fork that adds +# LWS to CSS 8.0.0-alpha.3 (pull request 126 there), at a commit +# (CSS_LWS_SHA), as no release or image of it is published. It is built +# without its dependencies' install scripts, then run with css-lws-e2e.json. +FROM node:22.23.4-bookworm-slim@sha256:4b150b48b4116e2a22f2c39cc27025fbf4f153e2078fd3b14d159fdb5c7279da AS build +# Renovate proposes newer commits of the branch for a person to approve (renovate.json5). +# renovate: datasource=git-refs depName=https://github.com/jeswr/CommunitySolidServer branch=feat/lws-next-major +ARG CSS_LWS_SHA=311b3677154256eb2f87add7dad50f1255cf50a3 +ARG CSS_LWS_VERSION=8.0.0-alpha.3 +RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates git && rm -rf /var/lib/apt/lists/* +WORKDIR /community-server +RUN git init -q . \ + && git fetch -q --depth 1 https://github.com/jeswr/CommunitySolidServer "$CSS_LWS_SHA" && git checkout -q FETCH_HEAD \ + && test "$(git rev-parse HEAD)" = "$CSS_LWS_SHA" \ + && test "$(node -p 'require("./package.json").version')" = "$CSS_LWS_VERSION" \ + && npm ci --ignore-scripts --no-audit --no-fund \ + && npm run build \ + && npm prune --omit=dev --ignore-scripts --no-audit --no-fund + +FROM node:22.23.4-bookworm-slim@sha256:4b150b48b4116e2a22f2c39cc27025fbf4f153e2078fd3b14d159fdb5c7279da +WORKDIR /community-server +COPY --from=build /community-server/package.json ./ +COPY --from=build /community-server/bin ./bin +COPY --from=build /community-server/config ./config +COPY --from=build /community-server/dist ./dist +COPY --from=build /community-server/node_modules ./node_modules +COPY --from=build /community-server/templates ./templates +COPY css-lws-e2e.json ./config/css-lws-e2e.json +USER node +ENTRYPOINT ["node", "bin/server.js"] diff --git a/e2e/pod/servers/css-lws/compose.yml b/e2e/pod/servers/css-lws/compose.yml new file mode 100644 index 000000000..36fa273fc --- /dev/null +++ b/e2e/pod/servers/css-lws/compose.yml @@ -0,0 +1,21 @@ +# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +# +# SPDX-License-Identifier: MIT + +# The Community Solid Server 8 with Linked Web Storage, built from +# ./Dockerfile, serving both the Solid Protocol and LWS, in memory, its +# root open to anyone (css-lws-e2e.json): an advisory server +# (docs/testing.md). The harness (../../servers.ts) sets E2E_PORT, the +# port on 127.0.0.1 it is reached at. +services: + server: + build: . + image: localhost/solid-memo-e2e-css-lws:local + pull_policy: never + command: ["-c", "config/css-lws-e2e.json", "-b", "http://127.0.0.1:${E2E_PORT:?}/", "-l", "warn"] + ports: ["127.0.0.1:${E2E_PORT:?}:3000"] + init: true + cap_drop: [ALL] + security_opt: ["no-new-privileges:true"] + mem_limit: 1g + pids_limit: 512 diff --git a/e2e/pod/servers/css-lws/css-lws-e2e.json b/e2e/pod/servers/css-lws/css-lws-e2e.json new file mode 100644 index 000000000..6add02a9e --- /dev/null +++ b/e2e/pod/servers/css-lws/css-lws-e2e.json @@ -0,0 +1,38 @@ +{ + "comment": "The configuration of the fork's config/solid-lws.json (the Solid Protocol and Linked Web Storage side by side, with WAC), in memory with the root as the storage, as the Community Solid Server's config/default.json has them. The root is initialized empty and open to anyone, as the end-to-end tests need (docs/testing.md).", + "@context": "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^8.0.0/components/context.jsonld", + "import": [ + "css:config/app/init/initialize-root.json", + "css:config/app/main/default.json", + "css:config/app/variables/default.json", + "css:config/http/handler/lws.json", + "css:config/http/middleware/default.json", + "css:config/http/notifications/all.json", + "css:config/http/server-factory/http.json", + "css:config/http/static/default.json", + "css:config/identity/access/public.json", + "css:config/identity/email/default.json", + "css:config/identity/handler/default.json", + "css:config/identity/lws/default.json", + "css:config/identity/oidc/default.json", + "css:config/identity/ownership/token.json", + "css:config/identity/pod/static.json", + "css:config/ldp/authentication/solid-lws.json", + "css:config/ldp/authorization/webacl.json", + "css:config/ldp/handler/solid-lws.json", + "css:config/ldp/metadata-parser/lws.json", + "css:config/ldp/metadata-writer/solid-lws.json", + "css:config/ldp/modes/lws.json", + "css:config/storage/backend/memory.json", + "css:config/storage/key-value/resource-store.json", + "css:config/storage/location/root.json", + "css:config/storage/middleware/lws.json", + "css:config/util/auxiliary/acl.json", + "css:config/util/identifiers/suffix.json", + "css:config/util/index/default.json", + "css:config/util/logging/winston.json", + "css:config/util/representation-conversion/solid-lws.json", + "css:config/util/resource-locker/memory.json", + "css:config/util/variables/default.json" + ] +} From 9c46c77351a5619b4ec771ed59994ea0ca5db44c Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Thu, 5 Nov 2026 19:15:58 +0100 Subject: [PATCH 02/19] build(deps): hold each commit of the CSS LWS fork for a person css-lws builds a commit of a fork nobody has reviewed, on CI's runners and on contributors' machines. Renovate follows the fork's branch, but a person approves each newer commit on the Dependency Dashboard before it is proposed, and merges it after reading the change. Its image keeps to Node 22, which the fork's own Dockerfile runs. Refs: #594 --- renovate.json5 | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/renovate.json5 b/renovate.json5 index beb7165f9..08eea5c08 100644 --- a/renovate.json5 +++ b/renovate.json5 @@ -136,6 +136,16 @@ '# renovate: datasource=(?\\S+) depName=(?\\S+) branch=(?\\S+)\\nARG SOLID_NEXTCLOUD_SHA=(?[0-9a-f]{40})', ], }, + // The commit of the Community Solid Server fork with Linked Web + // Storage its end-to-end tests' image builds (css-lws), as the + // fork's branch moves on: a person approves each (below). + { + customType: 'regex', + managerFilePatterns: ['/^e2e/pod/servers/css-lws/Dockerfile$/'], + matchStrings: [ + '# renovate: datasource=(?\\S+) depName=(?\\S+) branch=(?\\S+)\\nARG CSS_LWS_SHA=(?[0-9a-f]{40})', + ], + }, ], packageRules: [ // A new version of a package.json dependency raises its range to @@ -210,6 +220,22 @@ matchDepNames: ['node'], allowedVersions: '<23', }, + // The fork with Linked Web Storage is unreviewed code, built in CI + // and on contributors' machines: a person approves each new commit + // on the Dependency Dashboard before Renovate proposes it, and + // merges it after reading what it changes. Its image keeps to Node + // 22, which the fork's own Dockerfile runs. + { + matchFileNames: ['e2e/pod/servers/css-lws/Dockerfile'], + matchDepNames: ['https://github.com/jeswr/CommunitySolidServer'], + dependencyDashboardApproval: true, + automerge: false, + }, + { + matchFileNames: ['e2e/pod/servers/css-lws/Dockerfile'], + matchDepNames: ['node'], + allowedVersions: '<23', + }, // Solid-Nextcloud's app is tested on Nextcloud 31 to 35, but only 31 // and 32 must pass its CI. { From b5ff6e796ae1c61ed193912256a4360667dbac30 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Thu, 5 Nov 2026 21:13:20 +0100 Subject: [PATCH 03/19] feat(solid): find a Linked Web Storage storage as its server links it A server that speaks Linked Web Storage (LWS) alone names a resource's storage with a Link of relation lws#storage on every answer, and the storage's description (application/lws+cid) names its root container. It doesn't type the storage pim:Storage, and the Community Solid Server with LWS answers its storage's URI, its root container's too, with that description when asked for anything, which has none of the Solid links. So the app found no storage there, couldn't make a type index, and couldn't make a release public: the ACL lookup never reached the root's ACL. The storage walk-up takes an LWS storage's roots from its description when the WebID's document is in one, and a deck's folder is a storage's root only when the description names it. Each HEAD that reads a resource's links asks for Turtle first, or anything else, so the root keeps its Solid links where the server has Turtle, and a server that lists folders only as LWS has them still answers. Refs: #594 --- packages/solid-memo-client/src/access.ts | 26 ++--- packages/solid/src/client/access.test.ts | 11 +++ packages/solid/src/client/access.ts | 17 +++- packages/solid/src/client/linkHeader.test.ts | 34 +++++++ packages/solid/src/client/linkHeader.ts | 27 ++++++ packages/solid/src/deckPlace.test.ts | 58 ++++++++++++ packages/solid/src/deckPlace.ts | 17 +++- packages/solid/src/lwsStorage.test.ts | 94 +++++++++++++++++++ packages/solid/src/lwsStorage.ts | 73 ++++++++++++++ .../solid/src/solidStorageGateway.test.ts | 63 +++++++++++++ packages/solid/src/solidStorageGateway.ts | 39 +++++--- 11 files changed, 421 insertions(+), 38 deletions(-) create mode 100644 packages/solid/src/lwsStorage.test.ts create mode 100644 packages/solid/src/lwsStorage.ts diff --git a/packages/solid-memo-client/src/access.ts b/packages/solid-memo-client/src/access.ts index afb410d2c..6eb439860 100644 --- a/packages/solid-memo-client/src/access.ts +++ b/packages/solid-memo-client/src/access.ts @@ -5,7 +5,7 @@ import type { Quad, Term } from "@rdfjs/types"; import { DataFactory, Writer } from "n3"; import type { GrantPublicRead } from "@solid-memo/solid/client/interface"; -import { boundedHeader } from "@solid-memo/solid/client/linkHeader"; +import { linkTargets, TURTLE_FIRST } from "@solid-memo/solid/client/linkHeader"; import { parseQuads } from "./resources"; export type { GrantPublicRead } from "@solid-memo/solid/client/interface"; @@ -36,7 +36,7 @@ const ACCESS_CONTROL_RESOURCE = `${ACP}AccessControlResource`; * (solidReleasePublisher.ts) can read where each goes. */ export const grantPublicRead: GrantPublicRead = async (url, { fetch }) => { - const control = linked(await headOrGet(url, fetch), url, "acl")[0]; + const control = linkTargets(await headOrGet(url, fetch), "acl", url)[0]; if (control === undefined) return false; const current = await readControl(control, fetch); if (current === undefined) return false; @@ -81,7 +81,7 @@ async function readControl( fetch: typeof globalThis.fetch, ): Promise { const response = await fetch(url, { headers: { Accept: "text/turtle" } }); - const typed = linked(response, url, "type").includes(ACCESS_CONTROL_RESOURCE); + const typed = linkTargets(response, "type", url).includes(ACCESS_CONTROL_RESOURCE); if (response.status === 404) return { acp: typed, exists: false, quads: [] }; if (!response.ok) return undefined; const quads = await parseQuads(await response.text(), "turtle", base(response, url)); @@ -107,7 +107,7 @@ async function inheritedRules( let resource = new URL(url); while (resource.pathname !== "/") { const folder = new URL(resource.pathname.endsWith("/") ? ".." : ".", resource); - const folderAcl = linked(await headOrGet(folder.href, fetch), folder.href, "acl")[0]; + const folderAcl = linkTargets(await headOrGet(folder.href, fetch), "acl", folder.href)[0]; if (folderAcl === undefined) return undefined; const current = await readControl(folderAcl, fetch); if (current === undefined || current.acp) return undefined; @@ -258,20 +258,10 @@ async function insert( /** A HEAD of `url`, or a GET when the server will not answer that. */ async function headOrGet(url: string, fetch: typeof globalThis.fetch): Promise { - const head = await fetch(url, { method: "HEAD" }); - return head.ok ? head : fetch(url); -} - -/** The targets of a response's Link header with relation `rel`, made absolute; none when the header is past its bound (linkHeader.ts). */ -function linked(response: Response, url: string, rel: string): string[] { - const header = boundedHeader(response, "Link") ?? ""; - const found: string[] = []; - for (const [, target, params] of header.matchAll(/<([^>]*)>([^<]*)/g)) { - const relation = /;\s*rel\s*=\s*(?:"([^"]*)"|([^\s;,]+))/i.exec(params); - const relations = (relation?.[1] ?? relation?.[2] ?? "").split(/\s+/); - if (relations.includes(rel)) found.push(new URL(target, base(response, url)).href); - } - return found; + // Turtle asked for, as the app reads a folder: a Linked Web Storage server answers a storage's root otherwise, with its description, which links no ACL (docs/lws.md). + const headers = { Accept: TURTLE_FIRST }; + const head = await fetch(url, { method: "HEAD", headers }); + return head.ok ? head : fetch(url, { headers }); } function base(response: Response, url: string): string { diff --git a/packages/solid/src/client/access.test.ts b/packages/solid/src/client/access.test.ts index bd287eb11..34e85aa53 100644 --- a/packages/solid/src/client/access.test.ts +++ b/packages/solid/src/client/access.test.ts @@ -26,6 +26,17 @@ describe("accessFetch", () => { expect(fetch).toHaveBeenLastCalledWith(RELEASE, { method: "PUT", body: "<#a> <#b> <#c>." }); }); + it("asks for Turtle on a read that asks for nothing, as the app reads a folder, and leaves an Accept given be", async () => { + const fetch = vi.fn(async () => new Response(null)); + await accessFetch(fetch)(RELEASE, { method: "HEAD" }); + expect(fetch).toHaveBeenLastCalledWith(RELEASE, { + method: "HEAD", + headers: { accept: "text/turtle, */*;q=0.1" }, + }); + await accessFetch(fetch)(RELEASE, { headers: { Accept: "application/ld+json" } }); + expect(fetch).toHaveBeenLastCalledWith(RELEASE, { headers: { Accept: "application/ld+json" } }); + }); + it("reads an Access Control Resource not written yet as one that says only what it is, and leaves every other answer be", async () => { const ACR = `${RELEASE}.acr`; const type = '; rel="type"'; diff --git a/packages/solid/src/client/access.ts b/packages/solid/src/client/access.ts index 7598c51f6..bd2d4a22a 100644 --- a/packages/solid/src/client/access.ts +++ b/packages/solid/src/client/access.ts @@ -4,7 +4,7 @@ import { universalAccess } from "@inrupt/solid-client"; import type { GrantPublicRead } from "./interface"; -import { withBoundedHeaders } from "./linkHeader"; +import { TURTLE_FIRST, withBoundedHeaders } from "./linkHeader"; export type { GrantPublicRead } from "./interface"; @@ -59,6 +59,9 @@ const ACP_RESOURCE = `${ACP}resource`; * it (`<…#Agent>.}`), which node-solid-server's parser takes for a * path and refuses (400), as it does every PATCH of the library's * (sparqlUpdate.ts); an ACL holds no other kind of term; + * - a read that names no media type asks for Turtle: a Linked Web + * Storage server that is a storage's root too answers it otherwise + * with the storage's description, which links no ACL; * - for the Community Solid Server, an Access Control Resource not * written yet, which it answers 404 (its Link saying what it is), is * read as one that says only that it is one, which the write then @@ -83,10 +86,18 @@ export function accessFetch( }), ); } - const response = withBoundedHeaders(await fetch(input, init)); - const link = response.headers.get("Link") ?? ""; const method = init?.method ?? "GET"; const read = method === "GET" || method === "HEAD"; + // A read that asks for nothing asks for Turtle, as the app reads a folder: a Linked Web Storage server answers a storage's root otherwise, without the link to its ACL (docs/lws.md). + const asked = + read && !new Headers(init?.headers).has("Accept") + ? { + ...init, + headers: { ...Object.fromEntries(new Headers(init?.headers)), accept: TURTLE_FIRST }, + } + : init; + const response = withBoundedHeaders(await fetch(input, asked)); + const link = response.headers.get("Link") ?? ""; const isAcr = link.includes(`<${ACCESS_CONTROL_RESOURCE}>`); if (read && isAcr) onAcr(String(input)); if (!read || response.status !== 404 || !isAcr) return response; diff --git a/packages/solid/src/client/linkHeader.test.ts b/packages/solid/src/client/linkHeader.test.ts index bf0f78b95..a00b9b56b 100644 --- a/packages/solid/src/client/linkHeader.test.ts +++ b/packages/solid/src/client/linkHeader.test.ts @@ -7,6 +7,7 @@ import { getSolidDatasetLinear } from "./linearDataset"; import { boundedHeader, boundedHeadersFetch, + linkTargets, MAX_DESCRIBING_HEADER_LENGTH, withBoundedHeaders, } from "./linkHeader"; @@ -91,3 +92,36 @@ describe("getSolidDatasetLinear, with a long header", () => { expect(dataset.internal_resourceInfo.permissions?.user.read).toBe(true); }); }); + +describe("linkTargets", () => { + const at = (link: string, url = "https://pod.example/a/b") => { + const response = new Response(null, { headers: { Link: link } }); + Object.defineProperty(response, "url", { value: url }); + return response; + }; + + it("names the targets with the relation, made absolute against the response's URL", () => { + const response = at( + '<../s/>; rel="https://www.w3.org/ns/lws#storage", ; rel="type up", ; rel=acl, ; title="no rel"', + ); + expect( + linkTargets(response, "https://www.w3.org/ns/lws#storage", "https://other.example/"), + ).toEqual(["https://pod.example/s/"]); + expect(linkTargets(response, "up", "https://other.example/")).toEqual([ + "https://pod.example/a/x", + ]); + expect(linkTargets(response, "acl", "https://other.example/")).toEqual([ + "https://pod.example/a/y", + ]); + expect(linkTargets(response, "describedby", "https://other.example/")).toEqual([]); + }); + + it("resolves against the address asked for a response made here, and reads no header past its bound", () => { + expect(linkTargets(at("; rel=up", ""), "up", "https://pod.example/c/")).toEqual([ + "https://pod.example/c/x", + ]); + expect(linkTargets(new Response(null), "up", "https://pod.example/")).toEqual([]); + const long = `; rel=up, <${"y".repeat(MAX_DESCRIBING_HEADER_LENGTH)}>; rel=acl`; + expect(linkTargets(at(long), "up", "https://pod.example/")).toEqual([]); + }); +}); diff --git a/packages/solid/src/client/linkHeader.ts b/packages/solid/src/client/linkHeader.ts index 113c1de29..72484f521 100644 --- a/packages/solid/src/client/linkHeader.ts +++ b/packages/solid/src/client/linkHeader.ts @@ -51,3 +51,30 @@ export function withBoundedHeaders(response: Response): Response { export function boundedHeadersFetch(fetch: typeof globalThis.fetch): typeof globalThis.fetch { return async (input, init) => withBoundedHeaders(await fetch(input, init)); } + +/** + * The targets of a response's `Link` header with relation `rel`, made + * absolute against the response's URL (or `url`, for a response made + * here, which has none); none when the header is past its bound. A link + * may name several relations (`rel="type up"`), quoted or not. + */ +export function linkTargets(response: Response, rel: string, url: string): string[] { + const header = boundedHeader(response, "Link") ?? ""; + const found: string[] = []; + for (const [, target, params] of header.matchAll(/<([^>]*)>([^<]*)/g)) { + const relation = /;\s*rel\s*=\s*(?:"([^"]*)"|([^\s;,]+))/i.exec(params!); + const relations = (relation?.[1] ?? relation?.[2] ?? "").split(/\s+/); + if (relations.includes(rel)) found.push(new URL(target!, response.url || url).href); + } + return found; +} + +/** + * What a request that reads a resource's links asks for: Turtle, as the + * app reads a folder, or else whatever the server has. A Linked Web + * Storage server answers a storage's root, asked for anything, with the + * storage's description, which has none of the Solid Protocol's links + * (`pim:Storage`, the ACL's); one that lists a container only as LWS has + * it would answer Turtle alone with 406 (docs/lws.md). + */ +export const TURTLE_FIRST = "text/turtle, */*;q=0.1"; diff --git a/packages/solid/src/deckPlace.test.ts b/packages/solid/src/deckPlace.test.ts index 6e11ef5d7..860db3d35 100644 --- a/packages/solid/src/deckPlace.test.ts +++ b/packages/solid/src/deckPlace.test.ts @@ -703,6 +703,64 @@ describe("an instance at the pod's root, laid out flat", () => { await expect(busy.unclaimed(at(BOB))).resolves.toEqual([]); expect(heads).toBe(2); }); + + it("is not its own folder at the root a Linked Web Storage server names, and is below it", async () => { + const p = await pod(); + const at = (instance: string) => ({ + url: `${instance}catalog.ttl#deck-1`, + title: { en: "Spanish" }, + cardsDocumentUrl: `${instance}decks/deck-1.ttl`, + reviewsDocumentUrl: `${instance}reviews/deck-1.ttl`, + }); + const lws = (folder: string, root: string) => + (async (input, init) => { + if (String(input) === folder && init?.method === "HEAD") + return new Response(null, { + headers: { Link: `<${ROOT}bob/>; rel="https://www.w3.org/ns/lws#storage"` }, + }); + if (String(input) === `${ROOT}bob/` && init?.method === undefined) + return Response.json({ + id: `${ROOT}bob/`, + type: "Storage", + service: [{ type: "StorageRoot", serviceEndpoint: root }], + }); + return p.fetch(input, init); + }) as typeof globalThis.fetch; + await p.put(`${ROOT}bob/decks/deck-1.ttl`, ""); + await expect( + createDeckDocumentGuard(lws(`${ROOT}bob/`, `${ROOT}bob/`)).unclaimed(at(`${ROOT}bob/`)), + ).resolves.toEqual([`${ROOT}bob/decks/deck-1.ttl`]); + await p.put(`${BOB}decks/deck-1.ttl`, ""); + await expect( + createDeckDocumentGuard(lws(BOB, `${ROOT}bob/`)).unclaimed(at(BOB)), + ).resolves.toEqual([]); + }); + + it("asks again a folder whose storage description can't be read", async () => { + const p = await pod(); + let heads = 0; + const guard = createDeckDocumentGuard((async (input, init) => { + if (String(input) === BOB && init?.method === "HEAD") { + heads += 1; + return new Response(null, { + headers: { Link: `<${ROOT}bob/>; rel="https://www.w3.org/ns/lws#storage"` }, + }); + } + if (String(input) === `${ROOT}bob/` && init?.method === undefined) + return new Response(null, { status: 500 }); + return p.fetch(input, init); + }) as typeof globalThis.fetch); + await p.put(`${BOB}decks/deck-1.ttl`, ""); + const at = { + url: `${BOB}catalog.ttl#deck-1`, + title: { en: "Spanish" }, + cardsDocumentUrl: `${BOB}decks/deck-1.ttl`, + reviewsDocumentUrl: `${BOB}reviews/deck-1.ttl`, + }; + await expect(guard.unclaimed(at)).rejects.toMatchObject({ code: "cannotCheck" }); + await expect(guard.unclaimed(at)).rejects.toMatchObject({ code: "cannotCheck" }); + expect(heads).toBe(2); + }); }); describe("an instance whose address is spelled otherwise than a request sends it", () => { diff --git a/packages/solid/src/deckPlace.ts b/packages/solid/src/deckPlace.ts index 9a90ea4eb..85d0241db 100644 --- a/packages/solid/src/deckPlace.ts +++ b/packages/solid/src/deckPlace.ts @@ -29,6 +29,7 @@ import { RELEASE_DOCUMENT, } from "@solid-memo/domain/release/draftLayout"; import { documentUrlOf } from "@solid-memo/domain/subjectUrl"; +import { linkTargets, TURTLE_FIRST } from "./client/linkHeader"; import { getSolidDatasetOrNull, isReadFrom, @@ -36,6 +37,7 @@ import { saveDataset, UNREADABLE, } from "./datasets"; +import { LWS_STORAGE_REL, storageRootsOf } from "./lwsStorage"; import { deckBackLinksOf } from "./ownership"; import { hasStorageLink } from "./solidStorageGateway"; import { DCTERMS, RDF, SM } from "./vocab"; @@ -141,11 +143,22 @@ export function createDeckDocumentGuard(fetch: typeof globalThis.fetch) { if (known === undefined) { known = (async () => { if (new URL(folder).pathname === "/") return false; - const response = await fetch(folder, { method: "HEAD" }); + // Turtle asked for, as the app reads a folder: an LWS server may answer a storage's root otherwise, without its Solid links. + const response = await fetch(folder, { + method: "HEAD", + headers: { Accept: TURTLE_FIRST }, + }); // An answer that says nothing of the folder (401, 429, 503) is asked again, never kept as "not its own". if (!response.ok) throw new AppError("cannotCheck", { url: folder, status: response.status }); - return !hasStorageLink(response.headers.get("Link")); + if (hasStorageLink(response.headers.get("Link"))) return false; + // A Linked Web Storage server names a storage's root in its description (lwsStorage.ts), not with a type. + const [storage] = linkTargets(response, LWS_STORAGE_REL, folder); + if (storage === undefined) return true; + const roots = await storageRootsOf(storage, fetch); + // A description that can't be read says nothing of the folder: asked again, never kept as its own. + if (roots === null) throw new AppError("cannotCheck", { url: folder, status: 0 }); + return !roots.some((root) => isSameUrl(root, folder)); })(); known.catch(() => dedicated.delete(folder)); dedicated.set(folder, known); diff --git a/packages/solid/src/lwsStorage.test.ts b/packages/solid/src/lwsStorage.test.ts new file mode 100644 index 000000000..39b32f19b --- /dev/null +++ b/packages/solid/src/lwsStorage.test.ts @@ -0,0 +1,94 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { describe, expect, it } from "vitest"; +import { LWS_DESCRIPTION_TYPE, storageRootsOf } from "./lwsStorage"; + +const STORAGE = "https://pod.example/alice/"; + +/** A fetch that answers every request with `body` (JSON unless a string). */ +function answering(body: unknown, init?: ResponseInit): typeof globalThis.fetch { + return async () => + typeof body === "string" ? new Response(body, init) : Response.json(body, init); +} + +const described = (rest: Record) => ({ + "@context": ["https://www.w3.org/ns/cid/v1", "https://www.w3.org/ns/lws/v1"], + id: STORAGE, + type: "Storage", + ...rest, +}); + +describe("storageRootsOf", () => { + it("asks for the storage description and names its StorageRoot services' endpoints", async () => { + const asked: [string, RequestInit | undefined][] = []; + const fetch: typeof globalThis.fetch = async (input, init) => { + asked.push([String(input), init]); + return Response.json( + described({ + service: [ + { type: "StorageRoot", serviceEndpoint: STORAGE }, + { type: ["https://www.w3.org/ns/lws#StorageRoot"], serviceEndpoint: "/alice/more" }, + { type: "StorageRoot", serviceEndpoint: STORAGE }, + { type: "NotificationService", serviceEndpoint: "https://pod.example/.notifications/" }, + ], + }), + ); + }; + await expect(storageRootsOf(STORAGE, fetch)).resolves.toEqual([ + STORAGE, + "https://pod.example/alice/more/", + ]); + expect(asked).toEqual([[STORAGE, { headers: { Accept: LWS_DESCRIPTION_TYPE } }]]); + }); + + it("takes a type list and the full IRIs of the types", async () => { + const fetch = answering( + described({ + type: ["https://www.w3.org/ns/lws#Storage", "Other"], + service: [{ type: "StorageRoot", serviceEndpoint: STORAGE }], + }), + ); + await expect(storageRootsOf(STORAGE, fetch)).resolves.toEqual([STORAGE]); + }); + + it("leaves out services that are not maps, have no endpoint, or a root on another origin", async () => { + const fetch = answering( + described({ + service: [ + null, + "StorageRoot", + { type: "StorageRoot" }, + { type: "StorageRoot", serviceEndpoint: 7 }, + { type: "StorageRoot", serviceEndpoint: "http://[" }, + { type: "StorageRoot", serviceEndpoint: "https://elsewhere.example/" }, + ], + }), + ); + await expect(storageRootsOf(STORAGE, fetch)).resolves.toEqual([]); + }); + + it.each([ + ["an answer that is not ok", answering(described({}), { status: 404 })], + ["one that is not JSON", answering("")], + ["JSON that is no map", answering("null")], + [ + "a description of another storage", + answering({ ...described({}), id: "https://pod.example/" }), + ], + ["one with no id", answering({ type: "Storage", service: [] })], + ["one that is not a storage's", answering(described({ type: "Container", service: [] }))], + ["one with no services", answering(described({ service: {} }))], + ["one past its read limit", answering("x".repeat(64 * 1024 + 1))], + [ + "a request that fails", + (async () => { + throw new Error("offline"); + }) as typeof globalThis.fetch, + ], + ["one whose id is no URL", answering({ ...described({ service: [] }), id: "http://[" })], + ])("can't name the roots for %s", async (_, fetch) => { + await expect(storageRootsOf(STORAGE, fetch)).resolves.toBeNull(); + }); +}); diff --git a/packages/solid/src/lwsStorage.ts b/packages/solid/src/lwsStorage.ts new file mode 100644 index 000000000..b2aa3776b --- /dev/null +++ b/packages/solid/src/lwsStorage.ts @@ -0,0 +1,73 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import type { ReadLimits } from "./client/interface"; +import { limitedFetch } from "./client/readLimits"; +import { ensureTrailingSlash } from "./urls"; + +/** + * How a Linked Web Storage (LWS) server names its storages + * (docs/lws.md): every answer to a GET or HEAD links the storage the + * resource is in, with this relation, and the storage's URI answers + * with its storage description, which names the storage's root + * container as its StorageRoot service. The Solid Protocol's + * `pim:Storage` type is not part of LWS. + */ +export const LWS_STORAGE_REL = "https://www.w3.org/ns/lws#storage"; + +/** The media type of an LWS storage description (a Controlled Identifier document). */ +export const LWS_DESCRIPTION_TYPE = "application/lws+cid"; + +/** A storage description names a few services; it is read under these limits, far below a profile's. */ +const DESCRIPTION_LIMITS: ReadLimits = { bytes: 64 * 1024, quads: 0, characters: 64 * 1024 }; + +const STORAGE_TYPES = new Set(["Storage", "https://www.w3.org/ns/lws#Storage"]); +const STORAGE_ROOT_TYPES = new Set(["StorageRoot", "https://www.w3.org/ns/lws#StorageRoot"]); + +/** A JSON-LD `type`, a string or a list of them, as a list. */ +function typesOf(value: unknown): unknown[] { + return Array.isArray(value) ? value : [value]; +} + +/** + * The root containers of the storage at `storage`, as its storage + * description lists them (its StorageRoot services), each with a + * trailing slash; null when it answers no description, a description + * of another storage (its `id` must be the storage's URI), or one that + * is not one. A root on another origin than the storage's is left out, + * and so is one that is no URL: the app treats a storage's root as + * part of the server it asked. + */ +export async function storageRootsOf( + storage: string, + fetch: typeof globalThis.fetch, +): Promise { + let description: unknown; + try { + const response = await limitedFetch(fetch, DESCRIPTION_LIMITS)(storage, { + headers: { Accept: LWS_DESCRIPTION_TYPE }, + }); + if (!response.ok) return null; + description = JSON.parse(await response.text()); + } catch { + return null; + } + if (typeof description !== "object" || description === null) return null; + const { id, type, service } = description as Record; + if (typeof id !== "string" || !URL.canParse(id, storage)) return null; + if (new URL(id, storage).href !== new URL(storage).href) return null; + if (!typesOf(type).some((one) => STORAGE_TYPES.has(one as string))) return null; + if (!Array.isArray(service)) return null; + const roots = service + .filter((one): one is Record => typeof one === "object" && one !== null) + .filter((one) => typesOf(one.type).some((each) => STORAGE_ROOT_TYPES.has(each as string))) + .map((one) => one.serviceEndpoint) + .filter( + (endpoint): endpoint is string => + typeof endpoint === "string" && URL.canParse(endpoint, storage), + ) + .map((endpoint) => ensureTrailingSlash(new URL(endpoint, storage).href)) + .filter((root) => new URL(root).origin === new URL(storage).origin); + return [...new Set(roots)]; +} diff --git a/packages/solid/src/solidStorageGateway.test.ts b/packages/solid/src/solidStorageGateway.test.ts index c77f4158d..5b5479896 100644 --- a/packages/solid/src/solidStorageGateway.test.ts +++ b/packages/solid/src/solidStorageGateway.test.ts @@ -16,6 +16,17 @@ const profileStorages = vi.fn(); const CARD = "https://alice.example/profile/card"; const WEBID = `${CARD}#me`; const STORAGE_LINK = '; rel="type"'; +const LWS_STORAGE = "https://www.w3.org/ns/lws#storage"; + +/** An LWS storage description of `id`, its root container at `root`. */ +function description(id: string, root: string) { + return { + "@context": ["https://www.w3.org/ns/cid/v1", "https://www.w3.org/ns/lws/v1"], + id, + type: "Storage", + service: [{ type: "StorageRoot", serviceEndpoint: root }], + }; +} function headResponse(linkHeader?: string): Response { const headers = new Headers(); @@ -142,7 +153,59 @@ describe("createSolidStorageGateway", () => { ]); expect(fetch).toHaveBeenCalledWith("https://alice.example/profile/card", { method: "HEAD", + headers: { Accept: "text/turtle, */*;q=0.1" }, + }); + }); + + it("takes the storage a Linked Web Storage server links, as its description names its root", async () => { + profileStorages.mockResolvedValue([]); + const asked: [string, RequestInit | undefined][] = []; + const fetch = vi.fn(async (input, init) => { + asked.push([String(input), init]); + if (init?.method === "HEAD") + return headResponse(`; rel="${LWS_STORAGE}"`); + return Response.json(description("https://alice.example/", "https://alice.example/")); + }); + const gateway = createSolidStorageGateway({ fetch, profileStorages }); + + await expect(gateway.discoverStorages(WEBID)).resolves.toEqual([ + { url: "https://alice.example/", source: "linkHeader" }, + ]); + expect(asked).toEqual([ + [CARD, { method: "HEAD", headers: { Accept: "text/turtle, */*;q=0.1" } }], + ["https://alice.example/", { headers: { Accept: "application/lws+cid" } }], + ]); + }); + + it("walks on past a linked storage whose description can't be read", async () => { + profileStorages.mockResolvedValue([]); + const fetch = vi.fn(async (input, init) => { + if (init?.method !== "HEAD") return new Response(null, { status: 500 }); + return String(input) === "https://alice.example/" + ? headResponse(STORAGE_LINK) + : headResponse(`; rel="${LWS_STORAGE}"`); }); + const gateway = createSolidStorageGateway({ fetch, profileStorages }); + + await expect(gateway.discoverStorages(WEBID)).resolves.toEqual([ + { url: "https://alice.example/", source: "linkHeader" }, + ]); + }); + + it("walks on past a linked storage whose roots the document is not in", async () => { + profileStorages.mockResolvedValue([]); + const fetch = vi.fn(async (input, init) => { + if (init?.method !== "HEAD") + return Response.json(description("https://alice.example/", "https://other.example/")); + return String(input) === "https://alice.example/" + ? headResponse(STORAGE_LINK) + : headResponse(`; rel="${LWS_STORAGE}"`); + }); + const gateway = createSolidStorageGateway({ fetch, profileStorages }); + + await expect(gateway.discoverStorages(WEBID)).resolves.toEqual([ + { url: "https://alice.example/", source: "linkHeader" }, + ]); }); it("skips candidates whose HEAD request fails", async () => { diff --git a/packages/solid/src/solidStorageGateway.ts b/packages/solid/src/solidStorageGateway.ts index 1e376c931..79eb8cab5 100644 --- a/packages/solid/src/solidStorageGateway.ts +++ b/packages/solid/src/solidStorageGateway.ts @@ -14,6 +14,9 @@ import { import { getSourceUrl, type ProfileStorages } from "./client/resources"; import { readDataset } from "./datasets"; import { createPlacesFetch, profilePlacesOf, readNothing } from "./profilePlaces"; +import { isInFolder } from "@solid-memo/domain/instanceLayout"; +import { linkTargets, TURTLE_FIRST } from "./client/linkHeader"; +import { LWS_STORAGE_REL, storageRootsOf } from "./lwsStorage"; import { ensureTrailingSlash } from "./urls"; import { AppError } from "@solid-memo/domain/appError"; @@ -57,11 +60,7 @@ export function createSolidStorageGateway({ if (fromProfile.length > 0) { return fromProfile.map((url) => ({ url, source: "profile" })); } - const fromLinkHeader = await walkUpToStorage(webId, fetch); - if (fromLinkHeader !== null) { - return [{ url: fromLinkHeader, source: "linkHeader" }]; - } - return []; + return (await walkUpToStorage(webId, fetch)).map((url) => ({ url, source: "linkHeader" })); }, async probeStorage(url): Promise { @@ -119,21 +118,31 @@ export function hasStorageLink(linkHeader: string | null): boolean { .some((link) => link.includes(`<${PIM_STORAGE_TYPE}>`) && /rel="?type"?/.test(link)); } -async function walkUpToStorage( - webId: string, - fetch: typeof globalThis.fetch, -): Promise { +/** + * The storage the WebID's document is in, as its server links it: the + * first of the document and the folders above it whose answer is typed + * `pim:Storage` (the Solid Protocol), or, from a Linked Web Storage + * server, which links the storage of everything it serves, the roots of + * that storage the document is in (lwsStorage.ts). None when no answer + * names one. + */ +async function walkUpToStorage(webId: string, fetch: typeof globalThis.fetch): Promise { for (const candidate of candidateStorageUrls(webId)) { - let linkHeader: string | null; + let response: Response; try { - const response = await fetch(candidate, { method: "HEAD" }); - linkHeader = response.headers.get("Link"); + // Turtle asked for, as the app reads a container: an LWS server may answer a storage's root otherwise, without its Solid links. + response = await fetch(candidate, { method: "HEAD", headers: { Accept: TURTLE_FIRST } }); } catch { continue; } - if (hasStorageLink(linkHeader)) { - return candidate; + if (hasStorageLink(response.headers.get("Link"))) return [candidate]; + const [storage] = linkTargets(response, LWS_STORAGE_REL, candidate); + if (storage !== undefined) { + const roots = ((await storageRootsOf(storage, fetch)) ?? []).filter((root) => + isInFolder(candidate, root), + ); + if (roots.length > 0) return roots; } } - return null; + return []; } From a66b830b0b80a3b957d67752dfe6767b6148939d Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Thu, 5 Nov 2026 21:13:20 +0100 Subject: [PATCH 04/19] feat(solid): make the app's requests as a Linked Web Storage server takes them LWS Protocol 1.0 doesn't ask a server to take three requests the app makes of a Solid pod: a folder's listing in Turtle, a document made with PUT at a name the app chooses, and an edit with SPARQL Update. A server that takes only LWS refuses them, and the app can't work. A bridge below the fences sends each request as the app makes it, and only when an LWS server refuses one makes it as LWS has it: the listing read as application/lws+json, page by page; the document POSTed to its folder by the name asked for, each missing folder above it first, and refused where the server names it otherwise, since an instance's addresses are the app's to choose; the edit applied here and written whole with If-Match, tried again where another write came first. A Solid server gets each request once, as before. Refs: #594 --- packages/composition/src/appUseCases.ts | 9 +- packages/solid/src/localPod.ts | 2 +- packages/solid/src/lwsBridge.test.ts | 443 ++++++++++++++++++++++++ packages/solid/src/lwsBridge.ts | 397 +++++++++++++++++++++ packages/solid/src/testing/lwsPod.ts | 192 ++++++++++ 5 files changed, 1039 insertions(+), 4 deletions(-) create mode 100644 packages/solid/src/lwsBridge.test.ts create mode 100644 packages/solid/src/lwsBridge.ts create mode 100644 packages/solid/src/testing/lwsPod.ts diff --git a/packages/composition/src/appUseCases.ts b/packages/composition/src/appUseCases.ts index 2e63620c8..444ed8d4d 100644 --- a/packages/composition/src/appUseCases.ts +++ b/packages/composition/src/appUseCases.ts @@ -6,6 +6,7 @@ import { createUseCases, type UseCases } from "@solid-memo/application/useCases" import { createPodFence } from "@solid-memo/solid/podFence"; import { createWriteFence } from "@solid-memo/solid/writeFence"; import { anonymousFetch } from "@solid-memo/solid/authFetch"; +import { lwsBridge } from "@solid-memo/solid/lwsBridge"; import { withReadDeadline } from "@solid-memo/solid/readDeadline"; import { createSolidSession } from "@solid-memo/solid/solidSession"; import { createShaclShapeValidator } from "@solid-memo/solid/shaclShapeValidator"; @@ -109,17 +110,19 @@ export function createAppUseCases({ * (docs/guest-mode.md); then to the guest's pod or, as the logged-in * user, to any other, following no redirect to an address the fences * never checked (authFetch.ts). A read of another server that gets no - * answer in time is given up, to be tried again (readDeadline.ts). + * answer in time is given up, to be tried again (readDeadline.ts). A + * request a Linked Web Storage server does not take as the app makes + * it is made as LWS has it (lwsBridge.ts, docs/lws.md). */ const writeFence = createWriteFence( routedFetch({ origin: GUEST_ORIGIN, local: guestFetch, - remote: withReadDeadline(session.fetch), + remote: withReadDeadline(lwsBridge(session.fetch)), }), ); /** A fetch as no one (authFetch.ts), whose reads have a deadline as the user's do. */ - const noOneFetch = withReadDeadline(anonymousFetch); + const noOneFetch = withReadDeadline(lwsBridge(anonymousFetch)); const podFetch = createPodFence(writeFence.fetch); /** The release publisher's alone, for a release's own access control: the pod fence lets it write a document's ACL or ACR. */ const shareFetch = createPodFence(writeFence.fetch, { accessControl: true }); diff --git a/packages/solid/src/localPod.ts b/packages/solid/src/localPod.ts index 805dcb321..bf6a77db9 100644 --- a/packages/solid/src/localPod.ts +++ b/packages/solid/src/localPod.ts @@ -190,7 +190,7 @@ async function parsed(url: string, turtle: string): Promise * The document's triples with the update applied, block by block, each * line once (TripleLines); null when it is not one this pod applies. */ -async function patched( +export async function patched( url: string, triples: readonly string[], update: string, diff --git a/packages/solid/src/lwsBridge.test.ts b/packages/solid/src/lwsBridge.test.ts new file mode 100644 index 000000000..8e9f2f4c6 --- /dev/null +++ b/packages/solid/src/lwsBridge.test.ts @@ -0,0 +1,443 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { describe, expect, it } from "vitest"; +import { listContainer } from "./containers"; +import { appendToDocument, deleteDataset, readDataset, saveDataset } from "./datasets"; +import { buildThing, createSolidDataset, createThing, getThing, setThing } from "./client/rdf"; +import { CREATION_REFUSED, lwsBridge } from "./lwsBridge"; +import { lwsPod } from "./testing/lwsPod"; + +const ROOT = "https://pod.example/alice/"; +const MAIN = `${ROOT}solid-memo/main/`; +const CATALOG = `${MAIN}catalog.ttl`; +const MONTH = `${MAIN}history/2026-11.ttl`; +const TITLE = "http://purl.org/dc/terms/title"; +const STORAGE_LINK = `<${ROOT}>; rel="https://www.w3.org/ns/lws#storage"`; + +function titled(url: string, title: string) { + return buildThing(createThing({ url })).addStringNoLocale(TITLE, title).build(); +} + +/** A fetch whose answers `answer` gives, each with the URL asked for; every request kept in `asked`. */ +function scripted( + answer: (url: string, init: RequestInit | undefined, n: number) => Response | Promise, +) { + const asked: string[] = []; + const fetch: typeof globalThis.fetch = async (input, init) => { + asked.push(`${init?.method ?? "GET"} ${String(input)}`); + const response = await answer(String(input), init, asked.length); + if (response.url === "") Object.defineProperty(response, "url", { value: String(input) }); + return response; + }; + return { fetch, asked }; +} + +const lws = (status: number, headers: Record = {}, body: string | null = null) => + new Response(body, { status, headers: { Link: STORAGE_LINK, ...headers } }); + +describe("the app's documents on a server that takes only Linked Web Storage", () => { + it("makes each folder and document with a POST of its name, edits them whole, lists and deletes them", async () => { + const pod = lwsPod({ root: ROOT }); + const fetch = lwsBridge(pod.fetch); + await saveDataset( + CATALOG, + setThing(createSolidDataset(), titled(`${CATALOG}#it`, "one")), + fetch, + ); + expect(pod.resources.get(CATALOG)?.body).toContain('"one"'); + expect(pod.requests.filter((request) => request.startsWith("POST"))).toEqual([ + `POST ${ROOT}`, + `POST ${ROOT}solid-memo/`, + `POST ${MAIN}`, + ]); + + const read = await readDataset(CATALOG, fetch); + const changed = buildThing(getThing(read, `${CATALOG}#it`)!) + .setStringNoLocale(TITLE, "two") + .build(); + await saveDataset(CATALOG, setThing(read, changed), fetch); + expect(pod.resources.get(CATALOG)?.body).toContain('"two"'); + expect(pod.resources.get(CATALOG)?.body).not.toContain('"one"'); + expect(pod.requests).toContain(`PATCH ${CATALOG}`); + expect(pod.requests.at(-1)).toBe(`PUT ${CATALOG}`); + + expect(await listContainer(MAIN, fetch)).toEqual([CATALOG]); + await deleteDataset(CATALOG, await readDataset(CATALOG, fetch), fetch); + expect(pod.resources.has(CATALOG)).toBe(false); + expect(await listContainer(MAIN, fetch)).toEqual([]); + }); + + it("appends to a document not there yet, and keeps every one of several appends made at once", async () => { + const pod = lwsPod({ root: ROOT }); + const fetch = lwsBridge(pod.fetch); + await appendToDocument(MONTH, titled(`${MONTH}#a`, "a"), fetch); + await Promise.all( + ["b", "c", "d"].map((name) => + appendToDocument(MONTH, titled(`${MONTH}#${name}`, name), fetch), + ), + ); + for (const name of ["a", "b", "c", "d"]) + expect(pod.resources.get(MONTH)?.body).toContain(`"${name}"`); + expect(await listContainer(`${MAIN}history/`, fetch)).toEqual([MONTH]); + }); + + it("reads a listing in pages, and one a server gives only in the LWS profile", async () => { + for (const options of [{ pageSize: 2 }, { profileTurtle: true }]) { + const pod = lwsPod({ root: ROOT, ...options }); + const fetch = lwsBridge(pod.fetch); + const documents = ["a", "b", "c", "d", "e"].map((name) => `${MAIN}${name}.ttl`); + for (const document of documents) + await saveDataset( + document, + setThing(createSolidDataset(), titled(`${document}#it`, "x")), + fetch, + ); + expect(await listContainer(MAIN, fetch)).toEqual(documents); + } + }); + + it("refuses a document the server would name otherwise, and leaves nothing of it", async () => { + const pod = lwsPod({ root: ROOT, ownNames: true }); + const fetch = lwsBridge(pod.fetch); + await expect( + saveDataset(CATALOG, setThing(createSolidDataset(), titled(`${CATALOG}#it`, "one")), fetch), + ).rejects.toMatchObject({ response: { status: CREATION_REFUSED } }); + expect([...pod.resources.keys()]).toEqual([ROOT]); + }); +}); + +describe("lwsBridge, on a Solid server", () => { + it("sends every request as it is when the server takes it, or is no LWS server", async () => { + const { fetch, asked } = scripted((_, init) => { + if (init?.method === "HEAD") return new Response(null); + if (init?.method === "PUT" || init?.method === "PATCH") + return new Response(null, { status: 404 }); + return new Response("", { headers: { "Content-Type": "text/turtle" } }); + }); + const bridged = lwsBridge(fetch); + expect((await bridged(MAIN)).status).toBe(200); + expect( + (await bridged(CATALOG, { method: "PUT", headers: { "If-None-Match": "*" }, body: "" })) + .status, + ).toBe(404); + expect( + ( + await bridged(CATALOG, { + method: "PATCH", + headers: { "Content-Type": "application/sparql-update" }, + body: "", + }) + ).status, + ).toBe(404); + expect((await bridged(CATALOG, { method: "DELETE" })).status).toBe(200); + expect((await bridged(CATALOG, { method: "PATCH", body: "" })).status).toBe(404); + const request = new Request(CATALOG); + expect((await bridged(request)).status).toBe(200); + expect(asked).toEqual([ + `GET ${MAIN}`, + `PUT ${CATALOG}`, + `HEAD ${MAIN}`, + `PATCH ${CATALOG}`, + `HEAD ${CATALOG}`, + `DELETE ${CATALOG}`, + `PATCH ${CATALOG}`, + "GET [object Request]", + ]); + }); + + it("asks nothing more of a server that takes a creation or an update, or refuses it for another reason", async () => { + for (const status of [201, 412, 403, 400]) { + const { fetch, asked } = scripted(() => new Response(null, { status })); + const bridged = lwsBridge(fetch); + await bridged(CATALOG, { method: "PUT", headers: { "If-None-Match": "*" }, body: "" }); + await bridged(CATALOG, { + method: "PATCH", + headers: { "Content-Type": "application/sparql-update" }, + body: "", + }); + expect(asked).toHaveLength(2); + } + }); + + it("is no LWS server when no folder up to the root is there, or there are too many", async () => { + const { fetch, asked } = scripted(() => new Response(null, { status: 404 })); + await lwsBridge(fetch)("https://pod.example/a/b.ttl", { + method: "PUT", + headers: { "If-None-Match": "*" }, + }); + expect(asked).toEqual([ + "PUT https://pod.example/a/b.ttl", + "HEAD https://pod.example/a/", + "HEAD https://pod.example/", + ]); + const deep = `https://pod.example/${"x/".repeat(40)}d.ttl`; + const many = scripted(() => new Response(null, { status: 404 })); + await lwsBridge(many.fetch)(deep, { method: "PUT", headers: { "If-None-Match": "*" } }); + expect(many.asked).toHaveLength(33); + // The origin's root itself has no container to be made in. + const root = scripted(() => new Response(null, { status: 404 })); + await lwsBridge(root.fetch)("https://pod.example/", { + method: "PUT", + headers: { "If-None-Match": "*" }, + }); + expect(root.asked).toHaveLength(1); + }); +}); + +describe("lwsBridge, its listing", () => { + const listing = (body: unknown, headers: Record = {}) => + lws(200, { "Content-Type": "application/lws+json", ...headers }, JSON.stringify(body)); + + it("keeps the members of the container alone, and fails a listing that is not one", async () => { + const { fetch } = scripted(() => + listing({ + items: [{ id: `${MAIN}a.ttl` }, { id: 7 }, null, { id: `${ROOT}other.ttl` }, { id: MAIN }], + }), + ); + expect(await (await lwsBridge(fetch)(MAIN)).text()).toBe( + `<${MAIN}> a , ;\n <${MAIN}a.ttl> .\n`, + ); + const broken = scripted(() => listing({ type: "Container" })); + await expect(lwsBridge(broken.fetch)(MAIN)).rejects.toThrow( + "is not the listing of a container", + ); + }); + + it("follows a listing's pages on its origin alone, each once, and no further than it reads", async () => { + const next = (url: string) => ({ Link: `<${url}>; rel="next"` }); + for (const [link, message] of [ + ["https://elsewhere.example/p2", "goes on past"], + [MAIN, "goes on past"], + ] as const) { + const { fetch } = scripted(() => listing({ items: [] }, next(link))); + await expect(lwsBridge(fetch)(MAIN)).rejects.toThrow(message); + } + const endless = scripted((_, __, n) => listing({ items: [] }, next(`${MAIN}?p=${n}`))); + await expect(lwsBridge(endless.fetch)(MAIN)).rejects.toThrow("goes on past"); + expect(endless.asked).toHaveLength(200); + const failing = scripted((_, __, n) => + n === 1 ? listing({ items: [] }, next(`${MAIN}?p=2`)) : lws(500), + ); + await expect(lwsBridge(failing.fetch)(MAIN)).rejects.toThrow("answered 500"); + }); + + it("refuses a listing of more members than it reads", async () => { + const items = Array.from({ length: 20_001 }, (_, n) => ({ id: `${MAIN}${n}.ttl` })); + const { fetch } = scripted(() => listing({ items })); + await expect(lwsBridge(fetch)(MAIN)).rejects.toThrow("lists more than 20000"); + }); + + it("names the listing by the address asked for when the answer has none, and passes on one of no type", async () => { + const bare: typeof globalThis.fetch = async () => + new Response(JSON.stringify({ items: [{ id: "a.ttl" }] }), { + headers: { "Content-Type": "application/lws+json" }, + }); + const response = await lwsBridge(bare)(MAIN); + expect(response.url).toBe(MAIN); + expect(await response.text()).toContain(`<${MAIN}a.ttl>`); + const untyped: typeof globalThis.fetch = async () => new Response("x"); + expect(await (await lwsBridge(untyped)(MAIN)).text()).toBe("x"); + }); + + it("asks for its listing a server that will not give Turtle, and gives its answer when it has none", async () => { + const { fetch, asked } = scripted((_, init, n) => + n === 1 ? lws(406) : init?.method === "HEAD" ? lws(200) : lws(404), + ); + expect((await lwsBridge(fetch)(MAIN, { headers: { Accept: "text/turtle" } })).status).toBe(404); + expect(asked).toEqual([`GET ${MAIN}`, `HEAD ${MAIN}`, `GET ${MAIN}`]); + const plain = scripted((_, __, n) => + n === 1 ? lws(406) : lws(200, { "Content-Type": "text/plain" }), + ); + expect((await lwsBridge(plain.fetch)(MAIN)).headers.get("Content-Type")).toBe("text/plain"); + }); + + it("asks a server that is no LWS server for nothing more, whatever it answers a listing", async () => { + const { fetch, asked } = scripted((_, init) => + init?.method === "HEAD" ? new Response(null) : new Response(null, { status: 406 }), + ); + expect((await lwsBridge(fetch)(MAIN)).status).toBe(406); + expect(asked).toEqual([`GET ${MAIN}`, `HEAD ${MAIN}`]); + }); + + it("writes a member whose address holds what an IRI may not as an IRI", async () => { + const { fetch } = scripted(() => + lws( + 200, + { "Content-Type": "application/lws+json" }, + JSON.stringify({ items: [{ id: `${MAIN}a|b^c.ttl` }] }), + ), + ); + expect(await (await lwsBridge(fetch)(MAIN)).text()).toContain(`<${MAIN}a%7Cb%5Ec.ttl>`); + }); +}); + +describe("lwsBridge, its creation", () => { + const creating = { + method: "PUT", + headers: { "If-None-Match": "*", "Content-Type": "text/turtle" }, + body: "<#a> <#b> <#c>.", + }; + + it("says why a folder above could not be made, and what the server answered a POST", async () => { + const forbidden = scripted((url, init) => + init?.method === "HEAD" && url === MAIN ? lws(403) : lws(404), + ); + expect((await lwsBridge(forbidden.fetch)(CATALOG, creating)).status).toBe(403); + const refused = scripted((url, init) => + init?.method === "POST" + ? lws(400) + : init?.method === "HEAD" && url === MAIN + ? lws(200) + : lws(404), + ); + expect((await lwsBridge(refused.fetch)(CATALOG, creating)).status).toBe(400); + const deep = `${ROOT}${"x/".repeat(40)}d.ttl`; + const tooDeep = scripted((url, init) => + init?.method === "HEAD" && url === ROOT ? lws(200) : lws(404), + ); + expect((await lwsBridge(tooDeep.fetch)(deep, creating)).status).toBe(404); + }); + + it("deletes nothing but what the POST made in the folder, and never tries again what it could not delete", async () => { + for (const [location, deleteStatus] of [ + [null, 204], + ["https://elsewhere.example/alice/x.ttl", 204], + [`${ROOT}x.ttl`, 204], + [`${MAIN}x.ttl`, 500], + ] as const) { + const { fetch, asked } = scripted((url, init) => { + const method = init?.method ?? "GET"; + if (method === "POST") return lws(201, location === null ? {} : { Location: location }); + if (method === "HEAD") return url === MAIN ? lws(200) : lws(404); + if (method === "DELETE") return lws(deleteStatus); + return lws(405); + }); + const response = await lwsBridge(fetch)(CATALOG, creating); + expect(response.status).toBe(502); + expect(response.url).toBe(CATALOG); + const deletes = asked.filter((request) => request.startsWith("DELETE")); + expect(deletes).toEqual(location === `${MAIN}x.ttl` ? [`DELETE ${MAIN}x.ttl`] : []); + } + }); + + it("asks for a name outside ASCII as the address spells it", async () => { + const pod = lwsPod({ root: ROOT }); + const document = `${ROOT}한국어/카드.ttl`; + await saveDataset( + document, + setThing(createSolidDataset(), titled(`${document}#it`, "x")), + lwsBridge(pod.fetch), + ); + expect(pod.resources.has(new URL(document).href)).toBe(true); + }); + + it("makes a folder asked for with a POST that links its type, with no body", async () => { + const pod = lwsPod({ root: ROOT }); + const response = await lwsBridge(pod.fetch)(`${MAIN}decks/`, { + method: "PUT", + headers: { + "If-None-Match": "*", + "Content-Type": "text/turtle", + Link: '; rel="type"', + }, + }); + expect(response.status).toBe(201); + expect(response.url).toBe(`${MAIN}decks/`); + expect(pod.resources.get(`${MAIN}decks/`)).toMatchObject({ body: "" }); + }); +}); + +describe("lwsBridge, its update", () => { + const PATCH = (body: string, headers: Record = {}) => ({ + method: "PATCH", + headers: { "Content-Type": "application/sparql-update", ...headers }, + body, + }); + const INSERT = `INSERT DATA { <${CATALOG}#a> <${TITLE}> "a" . };`; + + it("does not write an update the app does not make, or one of a document with no version", async () => { + const pod = lwsPod({ root: ROOT }); + const fetch = lwsBridge(pod.fetch); + expect((await fetch(CATALOG, PATCH("DELETE WHERE { ?s ?p ?o }"))).status).toBe(415); + await saveDataset( + CATALOG, + setThing(createSolidDataset(), titled(`${CATALOG}#it`, "one")), + fetch, + ); + expect((await fetch(CATALOG, PATCH("DELETE WHERE { ?s ?p ?o }"))).status).toBe(415); + const unversioned = scripted((_, init) => + (init?.method ?? "GET") === "GET" + ? lws(200, { "Content-Type": "text/turtle" }, "") + : lws(415), + ); + expect((await lwsBridge(unversioned.fetch)(CATALOG, PATCH(INSERT))).status).toBe(415); + }); + + it("says another write came first to an edit of a document as read that is gone, or changed", async () => { + const pod = lwsPod({ root: ROOT }); + const fetch = lwsBridge(pod.fetch); + expect((await fetch(CATALOG, PATCH(INSERT, { "If-Match": '"1"' }))).status).toBe(412); + await fetch(CATALOG, PATCH(INSERT)); + expect((await fetch(CATALOG, PATCH(INSERT, { "If-Match": '"0"' }))).status).toBe(412); + }); + + it("gives up after a few tries, and passes on a read that fails", async () => { + const busy = scripted((_, init) => + (init?.method ?? "GET") === "GET" + ? lws(200, { "Content-Type": "text/turtle", ETag: '"1"' }, "") + : init?.method === "PUT" + ? lws(412) + : lws(415), + ); + expect((await lwsBridge(busy.fetch)(CATALOG, PATCH(INSERT))).status).toBe(412); + expect(busy.asked.filter((request) => request.startsWith("PUT"))).toHaveLength(5); + const failing = scripted((_, init) => + (init?.method ?? "GET") === "GET" ? lws(403) : lws(415), + ); + expect((await lwsBridge(failing.fetch)(CATALOG, PATCH(INSERT))).status).toBe(403); + }); + + it("writes nothing against a weak ETag, and fails an update whose deletion is not there", async () => { + const weak = scripted((_, init) => + (init?.method ?? "GET") === "GET" + ? lws(200, { "Content-Type": "text/turtle", ETag: 'W/"1"' }, "") + : lws(415), + ); + expect((await lwsBridge(weak.fetch)(CATALOG, PATCH(INSERT))).status).toBe(415); + expect(weak.asked.some((request) => request.startsWith("PUT"))).toBe(false); + const pod = lwsPod({ root: ROOT }); + const fetch = lwsBridge(pod.fetch); + const DELETE = `DELETE DATA { <${CATALOG}#b> <${TITLE}> "b" . }; ${INSERT}`; + expect((await fetch(CATALOG, PATCH(DELETE))).status).toBe(409); + expect(pod.resources.has(CATALOG)).toBe(false); + await fetch(CATALOG, PATCH(INSERT)); + expect((await fetch(CATALOG, PATCH(DELETE))).status).toBe(409); + }); + + it("says another write came first, not that a deletion isn't there, to an edit of a document changed since it was read", async () => { + const pod = lwsPod({ root: ROOT }); + const fetch = lwsBridge(pod.fetch); + await fetch(CATALOG, PATCH(INSERT)); + const DELETE = `DELETE DATA { <${CATALOG}#b> <${TITLE}> "b" . }; ${INSERT}`; + expect((await fetch(CATALOG, PATCH(DELETE, { "If-Match": '"0"' }))).status).toBe(412); + }); + + it("changes the document another tab made first by the same name", async () => { + let made = false; + const { fetch } = scripted((url, init) => { + const method = init?.method ?? "GET"; + if (method === "PATCH") return lws(404); + if (method === "HEAD") return url === MAIN || url === `${MAIN}history/` ? lws(200) : lws(404); + if (method === "GET") + return made ? lws(200, { "Content-Type": "text/turtle", ETag: '"1"' }, "") : lws(404); + if (method === "POST") { + made = true; + return lws(201, { Location: `${MAIN}history/2026-11-2.ttl` }); + } + return lws(204); + }); + expect((await lwsBridge(fetch)(MONTH, PATCH(INSERT))).status).toBe(204); + }); +}); diff --git a/packages/solid/src/lwsBridge.ts b/packages/solid/src/lwsBridge.ts new file mode 100644 index 000000000..f19a9298c --- /dev/null +++ b/packages/solid/src/lwsBridge.ts @@ -0,0 +1,397 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { isInFolder, isSameUrl } from "@solid-memo/domain/instanceLayout"; +import type { ReadLimits } from "./client/interface"; +import { linkTargets, TURTLE_FIRST } from "./client/linkHeader"; +import { limitedFetch } from "./client/readLimits"; +import { dataBlocks, parentOf, patched } from "./localPod"; +import { LWS_STORAGE_REL } from "./lwsStorage"; +import { triplesOf } from "./ntriples"; + +/** + * The bridge between the Solid Protocol requests the app makes and a + * Linked Web Storage (LWS) server that takes only what LWS asks of it + * (docs/lws.md). Three requests of the app have no LWS equivalent; each + * is first sent as it is, and only when the server refuses it, and + * says it is an LWS server, made as LWS has it: + * + * - a container's listing in Turtle (`ldp:contains`): LWS lists a + * container as `application/lws+json`, page by page; the pages are + * read and given as the Turtle listing the app reads; + * - a document or container made with PUT and `If-None-Match: *`: LWS + * makes a resource with a POST to its container, which names it, and + * each container above it first; the name asked for is sent as a + * hint (`Slug`), and a resource the server names otherwise is deleted + * and the creation refused (CREATION_REFUSED); + * - a SPARQL Update PATCH (`INSERT DATA`, `DELETE DATA`): LWS requires + * only JSON Patch; the document is read, changed here, and written + * whole with `If-Match`, its ETag as read when the PATCH had none, + * tried again when another write came first; a document not there + * yet is made as above. + * + * Every other request, and every request to a Solid server, is sent as + * it is. It sits below the fences (podFence.ts, ownPodGate.ts), which + * check each request as the app makes it; what it sends besides is to + * the same resource, the containers above it, or the pages the server + * links. It imports no library. + */ +export function lwsBridge(fetch: typeof globalThis.fetch): typeof globalThis.fetch { + return async (input, init) => { + // The app asks by URL; a Request is sent as it is. + if (typeof input !== "string" && !(input instanceof URL)) return fetch(input, init); + const url = String(input); + const method = (init?.method ?? "GET").toUpperCase(); + const headers = new Headers(init?.headers); + if (method === "GET" && url.endsWith("/")) return listing(fetch, url, init); + if (method === "PUT" && headers.get("If-None-Match") === "*") + return creation(fetch, url, init!); + if (method === "PATCH" && mediaTypeOf(headers.get("Content-Type")) === SPARQL_UPDATE) + return update(fetch, url, init!); + return fetch(input, init); + }; +} + +/** The status of a creation the server refused to make by the name asked for: it names its resources itself. */ +export const CREATION_REFUSED = 501; + +/** The status of a creation the server made by another name, which the bridge could not delete again: never tried again. */ +const CREATION_LEFT = 502; + +const LWS_JSON = "application/lws+json"; +const LWS_PROFILE = "https://www.w3.org/ns/lws/v1"; +const LWS_CONTAINER = "https://www.w3.org/ns/lws#Container"; +const LDP = "http://www.w3.org/ns/ldp#"; +const SPARQL_UPDATE = "application/sparql-update"; +/** What a server answers a request whose method or media type it does not take. */ +const NOT_TAKEN = new Set([405, 415, 501]); +/** The most pages of a listing, and items, read: a listing past them is refused, as a container past the tree budget is (containers.ts). */ +const MAX_PAGES = 200; +const MAX_ITEMS = 20_000; +/** Each page of a listing is read under these limits. */ +const PAGE_LIMITS: ReadLimits = { bytes: 4 * 1024 * 1024, quads: 0, characters: 4 * 1024 * 1024 }; +/** The most containers above a resource made, or asked whether the server is an LWS one. */ +const MAX_DEPTH = 32; +/** The most times a PATCH without `If-Match` is tried again after another write came first. */ +const MAX_TRIES = 5; + +/** A media type without its parameters, in lower case; "" for none. */ +function mediaTypeOf(contentType: string | null): string { + return (contentType ?? "").split(";")[0]!.trim().toLowerCase(); +} + +/** A response for `url`, as a server's answer is: its URL set. */ +function named(response: Response, url: string): Response { + Object.defineProperty(response, "url", { value: url }); + return response; +} + +/** + * Whether the server of `url` is an LWS server: the first of `url` and + * the containers above it that is there links its storage + * (lwsStorage.ts), as every answer of an LWS server to a GET or HEAD + * does. + */ +async function isLws( + fetch: typeof globalThis.fetch, + url: string, + signal: AbortSignal | null | undefined, +): Promise { + let at: string | null = url; + for (let depth = 0; at !== null && depth < MAX_DEPTH; depth++) { + const response = await fetch(at, { + method: "HEAD", + headers: { Accept: TURTLE_FIRST }, + signal, + }); + if (response.status !== 404) return linkTargets(response, LWS_STORAGE_REL, at).length > 0; + at = parentOf(at); + } + return false; +} + +// The listing. + +async function listing( + fetch: typeof globalThis.fetch, + url: string, + init: RequestInit | undefined, +): Promise { + const response = await fetch(url, init); + const contentType = response.headers.get("Content-Type"); + if (response.ok && mediaTypeOf(contentType) === LWS_JSON) + return listingOf(fetch, url, init, response); + // An LWS server that gives Turtle in the LWS profile (lws:items, no ldp:contains), or none, is asked for its own listing. + const lwsProfile = response.ok && String(contentType).includes(LWS_PROFILE); + if (response.status !== 406 && !lwsProfile) return response; + if (!(await isLws(fetch, url, init?.signal))) return response; + await response.body?.cancel(); + const asked = await fetch(url, { ...init, headers: withAccept(init, LWS_JSON) }); + if (!asked.ok || mediaTypeOf(asked.headers.get("Content-Type")) !== LWS_JSON) return asked; + return listingOf(fetch, url, init, asked); +} + +function withAccept(init: RequestInit | undefined, accept: string): Headers { + const headers = new Headers(init?.headers); + headers.set("Accept", accept); + return headers; +} + +/** + * The listing of the container at `url` whose first page is `first` + * (`application/lws+json`), every page read, as the Turtle the app + * reads: the container, typed as one, and `ldp:contains` each resource + * in it. The first page's status and headers (its ETag, its links) + * are kept. A page that is not a container's listing, or past the + * limits, fails the read. + */ +async function listingOf( + fetch: typeof globalThis.fetch, + url: string, + init: RequestInit | undefined, + first: Response, +): Promise { + const pageFetch = limitedFetch(fetch, PAGE_LIMITS); + const items = new Set(); + const seen = new Set(); + let page = await limitedFetch(async () => first, PAGE_LIMITS)(url); + let at = first.url || url; + for (let pages = 1; ; pages++) { + const body: unknown = JSON.parse(await page.text()); + const listed = (body as { items?: unknown } | null)?.items; + if (!Array.isArray(listed)) throw new Error(`${at} is not the listing of a container.`); + for (const item of listed) { + const id = (item as { id?: unknown } | null)?.id; + if (typeof id !== "string") continue; + const member = new URL(id, at).href; + if (isInFolder(member, url) && !isSameUrl(member, url)) items.add(member); + } + if (items.size > MAX_ITEMS) throw new Error(`${url} lists more than ${MAX_ITEMS} resources.`); + seen.add(at); + const [next] = linkTargets(page, "next", at); + if (next === undefined) break; + if (new URL(next).origin !== new URL(url).origin || seen.has(next) || pages === MAX_PAGES) + throw new Error(`${url}'s listing goes on past what the app reads.`); + page = await pageFetch(next, { headers: { Accept: LWS_JSON }, signal: init?.signal }); + if (!page.ok) throw new Error(`${next} answered ${page.status}.`); + at = next; + } + const contains = [...items].map((member) => `<${iriOf(member)}>`).join(", "); + const turtle = `<${url}> a <${LDP}Container>, <${LDP}BasicContainer>${ + contains === "" ? "" : `;\n <${LDP}contains> ${contains}` + } .\n`; + const headers = new Headers(first.headers); + headers.set("Content-Type", "text/turtle"); + headers.delete("Content-Length"); + return named(new Response(turtle, { status: first.status, headers }), first.url || url); +} + +/** An address as a Turtle IRI: the characters an IRI may not hold, which a URL's path keeps (`|`, `^`), percent-encoded. */ +function iriOf(url: string): string { + return url.replace( + /[\u0000-\u0020<>"{}|^`\\]/g, + (c) => `%${c.charCodeAt(0).toString(16).toUpperCase().padStart(2, "0")}`, + ); +} + +// Creation. + +async function creation( + fetch: typeof globalThis.fetch, + url: string, + init: RequestInit, +): Promise { + const response = await fetch(url, init); + if (response.status !== 404 && !NOT_TAKEN.has(response.status)) return response; + const parent = parentOf(url); + if (parent === null || !(await isLws(fetch, parent, init.signal))) return response; + return post(fetch, url, init); +} + +/** + * The resource at `url` made as LWS makes one: each container above it + * that is not there made first, then a POST to its container, its name + * asked for. The answer is the POST's, its URL `url`; CREATION_REFUSED + * when the server named it otherwise (and it is deleted again). + */ +async function post( + fetch: typeof globalThis.fetch, + url: string, + init: RequestInit, + depth = 0, +): Promise { + const parent = parentOf(url)!; + const above = await containerThere(fetch, parent, init, depth); + if (above !== null) return above; + const container = url.endsWith("/"); + const headers = new Headers(init.headers); + // A creation's conditions mean nothing to a POST, and a type it links is the server's to set (LWS: server-managed). + headers.delete("If-None-Match"); + headers.delete("If-Match"); + headers.delete("Link"); + // The name as the address spells it, its escapes kept: a header's value is bytes (RFC 5023 has the server undo them). + headers.set( + "Slug", + new URL(url).pathname + .split("/") + .filter((segment) => segment !== "") + .at(-1)!, + ); + if (container) { + // A container: made with no body, its type linked, as LWS asks of the client. + headers.delete("Content-Type"); + headers.set("Link", `<${LWS_CONTAINER}>; rel="type"`); + } + const posted = await fetch(parent, { + method: "POST", + headers, + body: container ? undefined : init.body, + redirect: init.redirect, + signal: init.signal, + }); + if (!posted.ok) return posted; + const location = posted.headers.get("Location"); + const made = location === null ? null : new URL(location, parent).href; + if (made !== null && isSameUrl(made, url)) return named(posted, url); + // Only a resource the POST made in the container is deleted again: another address the answer names is no one's to delete here. + const madeHere = + made !== null && new URL(made).origin === new URL(parent).origin && parentOf(made) === parent; + const deleted = madeHere && (await fetch(made, { method: "DELETE", signal: init.signal })).ok; + return named( + deleted + ? new Response(null, { + status: CREATION_REFUSED, + statusText: "The server names its resources itself", + }) + : new Response(null, { + status: CREATION_LEFT, + statusText: "The server made the resource where it would not say, or could not delete it", + }), + url, + ); +} + +/** Null when the container at `url` is there, or once it is made (post); else the answer that says why it is not. */ +async function containerThere( + fetch: typeof globalThis.fetch, + url: string, + init: RequestInit, + depth: number, +): Promise { + const response = await fetch(url, { + method: "HEAD", + headers: { Accept: TURTLE_FIRST }, + signal: init.signal, + }); + if (response.ok) return null; + if (response.status !== 404 || depth >= MAX_DEPTH || parentOf(url) === null) return response; + const made = await post( + fetch, + url, + { method: "PUT", signal: init.signal, redirect: init.redirect }, + depth + 1, + ); + return made.ok ? null : made; +} + +// The update. + +async function update( + fetch: typeof globalThis.fetch, + url: string, + init: RequestInit, +): Promise { + const refused = await fetch(url, init); + if (refused.ok || (refused.status !== 404 && !NOT_TAKEN.has(refused.status))) return refused; + if (!(await isLws(fetch, url, init.signal))) return refused; + const edit: Edit = { + url, + init, + sparql: String(init.body), + condition: new Headers(init.headers).get("If-Match"), + refused, + }; + let last: Try = { answer: refused, again: false }; + for (let tries = 0; tries < MAX_TRIES && (tries === 0 || last.again); tries++) + last = await updateOnce(fetch, edit); + return last.answer; +} + +/** A SPARQL Update the server refused: the document, the request, the update, its If-Match, and the refusal. */ +interface Edit { + url: string; + init: RequestInit; + sparql: string; + condition: string | null; + refused: Response; +} + +/** The answer to one try of an update, and whether to try again: another write came first to an update with no If-Match. */ +interface Try { + answer: Response; + again: boolean; +} + +const conflict = (url: string) => named(new Response(null, { status: 409 }), url); + +/** The edit made once: the document read, changed here, and written whole with If-Match. */ +async function updateOnce(fetch: typeof globalThis.fetch, edit: Edit): Promise { + const { url, init, sparql, condition } = edit; + const current = await fetch(url, { + headers: { Accept: "text/turtle" }, + cache: "no-store", + signal: init.signal, + }); + if (current.status === 404) return madeBy(fetch, edit); + if (!current.ok) return { answer: current, again: false }; + // A weak ETag never matches If-Match (RFC 9110): a document with none strong has no version to write against. + const etag = current.headers.get("ETag"); + // An edit of the document as read, which has changed since: another write came first, as the PUT would say. + if (condition !== null && etag !== null && etag !== condition) + return { answer: named(new Response(null, { status: 412 }), url), again: false }; + const version = condition ?? (etag === null || etag.startsWith("W/") ? null : etag); + const read = new Set(await triplesOf(url, await current.text())); + const triples = await patched(url, [...read], sparql); + // An update the app does not make, or a document with no version to write it against, is not written. + if (triples === null || version === null) return { answer: edit.refused, again: false }; + // A triple to delete that is not there fails the update, as a Solid server's PATCH does (409). + if ([...(await deletionsOf(url, sparql))].some((triple) => !read.has(triple))) + return { answer: conflict(url), again: false }; + const written = await fetch(url, { + method: "PUT", + headers: { "Content-Type": "text/turtle", "If-Match": version }, + body: `${triples.join("\n")}\n`, + redirect: init.redirect, + signal: init.signal, + }); + return { answer: written, again: written.status === 412 && condition === null }; +} + +/** The edit of a document not there yet: the document made with what it inserts, unless it was an edit of one as read. */ +async function madeBy(fetch: typeof globalThis.fetch, edit: Edit): Promise { + const { url, init, sparql } = edit; + // An edit of a document as read finds none: another write came first. + if (edit.condition !== null) + return { answer: named(new Response(null, { status: 412 }), url), again: false }; + const triples = await patched(url, [], sparql); + if (triples === null) return { answer: edit.refused, again: false }; + if ((await deletionsOf(url, sparql)).size > 0) return { answer: conflict(url), again: false }; + const made = await post(fetch, url, { + ...init, + method: "PUT", + headers: { "Content-Type": "text/turtle" }, + body: `${triples.join("\n")}\n`, + }); + // Another tab made it first, by the same name: its document is changed instead. + return { answer: made, again: made.status === CREATION_REFUSED }; +} + +/** The triples an update's `DELETE DATA` blocks name, as N-Triples; an update `patched` applies only. */ +async function deletionsOf(url: string, sparql: string): Promise> { + const deletions = new Set(); + for (const { operation, data } of dataBlocks(sparql)!) + if (operation === "DELETE") + for (const triple of await triplesOf(url, data)) deletions.add(triple); + return deletions; +} diff --git a/packages/solid/src/testing/lwsPod.ts b/packages/solid/src/testing/lwsPod.ts new file mode 100644 index 000000000..fe2d5f54f --- /dev/null +++ b/packages/solid/src/testing/lwsPod.ts @@ -0,0 +1,192 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +/** + * A Linked Web Storage server, in memory, that takes only what LWS + * Protocol 1.0 asks of a server (docs/lws.md), for the tests of the + * bridge (lwsBridge.ts): a container is listed only as + * `application/lws+json` (Turtle refused with 406, or given in the LWS + * profile), page by page; a resource is made only by a POST to its + * container, which names it from the `Slug` it is given (or, when told + * so, by a name of its own); PUT replaces a resource that is there and + * answers 404 for one that is not; PATCH is refused (415), as this one + * takes no JSON Patch; every answer to a GET or HEAD links the storage. + */ +export interface LwsPodOptions { + /** The storage's URI, its root container's too: a URL ending in a slash. */ + root: string; + /** The most items on a page of a listing. */ + pageSize?: number; + /** Turtle in the LWS profile for a container's Turtle, not 406. */ + profileTurtle?: boolean; + /** Names a resource made by POST itself, the Slug left aside. */ + ownNames?: boolean; +} + +export interface LwsPod { + fetch: typeof globalThis.fetch; + /** The documents, by URL, and the containers (their URL ends in a slash, their body ""). */ + resources: Map; + /** Every request, as "METHOD url". */ + requests: string[]; +} + +const STORAGE_REL = "https://www.w3.org/ns/lws#storage"; + +export function lwsPod({ + root, + pageSize = 1000, + profileTurtle = false, + ownNames = false, +}: LwsPodOptions): LwsPod { + const resources = new Map(); + const requests: string[] = []; + let version = 0; + let named = 0; + const etag = () => `"${++version}"`; + /** A container's ETag changes with its members, as a listing's does. */ + const touch = (container: string) => { + const there = resources.get(container); + if (there !== undefined) resources.set(container, { ...there, etag: etag() }); + }; + resources.set(root, { body: "", type: "", etag: etag() }); + + const parentOf = (url: string) => { + const trimmed = url.endsWith("/") ? url.slice(0, -1) : url; + return trimmed.slice(0, trimmed.lastIndexOf("/") + 1); + }; + const members = (container: string) => + [...resources.keys()].filter((url) => url !== container && parentOf(url) === container).sort(); + const answer = ( + status: number, + body: string | null, + headers: Record, + url: string, + ) => { + const response = new Response(status === 204 || status === 304 ? null : body, { + status, + headers: { Link: `<${root}>; rel="${STORAGE_REL}"`, ...headers }, + }); + Object.defineProperty(response, "url", { value: url }); + return response; + }; + + /** What a request names: the resource (without a page's query), the address asked, its headers and body. */ + interface Asked { + url: string; + href: string; + method: string; + headers: Headers; + body: string; + } + + /** A container's listing, as LWS has it: a page of its members, linked to the next. */ + const listing = ({ url, href, method }: Asked, etagOf: string) => { + const all = members(url); + const page = Number(new URL(href).searchParams.get("page") ?? "0"); + const items = all.slice(page * pageSize, (page + 1) * pageSize).map((id) => { + const member = resources.get(id)!; + return id.endsWith("/") + ? { id, type: "Container" } + : { id, type: "DataResource", format: member.type, size: member.body.length }; + }); + const links = [`<${root}>; rel="${STORAGE_REL}"`]; + if (all.length > pageSize) links.push(`<${url}?page=0>; rel="first"`); + if ((page + 1) * pageSize < all.length) links.push(`<${url}?page=${page + 1}>; rel="next"`); + const body = JSON.stringify({ + "@context": "https://www.w3.org/ns/lws/v1", + id: url, + type: "Container", + totalItems: all.length, + items, + }); + return answer( + 200, + method === "HEAD" ? null : body, + { "Content-Type": "application/lws+json", ETag: etagOf, Link: links.join(", ") }, + href, + ); + }; + + const read = (asked: Asked) => { + const { url, href, method, headers } = asked; + const resource = resources.get(url); + if (resource === undefined) return answer(404, null, {}, href); + if (headers.get("If-None-Match") === resource.etag) + return answer(304, null, { ETag: resource.etag }, href); + const body = method === "HEAD" ? null : resource.body; + if (!url.endsWith("/")) + return answer(200, body, { "Content-Type": resource.type, ETag: resource.etag }, href); + const accept = headers.get("Accept") ?? "*/*"; + if (accept.includes("application/lws+json") || accept.includes("*/*")) + return listing(asked, resource.etag); + if (!profileTurtle) return answer(406, null, {}, href); + const profiled = method === "HEAD" ? null : `<> a .`; + return answer( + 200, + profiled, + { + "Content-Type": 'text/turtle; profile="https://www.w3.org/ns/lws/v1"', + ETag: resource.etag, + }, + href, + ); + }; + + const replace = ({ url, href, headers, body }: Asked) => { + const resource = resources.get(url); + if (resource === undefined) return answer(404, null, {}, href); + const ifMatch = headers.get("If-Match"); + if (headers.get("If-None-Match") === "*" || (ifMatch !== null && ifMatch !== resource.etag)) + return answer(412, null, {}, href); + resources.set(url, { body, type: headers.get("Content-Type") ?? "", etag: etag() }); + return answer(204, null, { ETag: resources.get(url)!.etag }, href); + }; + + const create = ({ url, href, headers, body }: Asked) => { + if (!resources.has(url) || !url.endsWith("/")) return answer(404, null, {}, href); + const container = (headers.get("Link") ?? "").includes("https://www.w3.org/ns/lws#Container"); + const slug = headers.get("Slug"); + const name = + ownNames || slug === null ? `r${++named}` : encodeURIComponent(decodeURIComponent(slug)); + const end = container ? "/" : ""; + const made = resources.has(`${url}${name}${end}`) + ? `${url}${name}-${++named}${end}` + : `${url}${name}${end}`; + resources.set(made, { + body: container ? "" : body, + type: container ? "" : (headers.get("Content-Type") ?? ""), + etag: etag(), + }); + touch(url); + return answer(201, null, { Location: made, ETag: resources.get(made)!.etag }, href); + }; + + const remove = ({ url, href }: Asked) => { + if (!resources.has(url)) return answer(404, null, {}, href); + if (url.endsWith("/") && members(url).length > 0) return answer(409, null, {}, href); + resources.delete(url); + touch(parentOf(url)); + return answer(204, null, {}, href); + }; + + const fetch: typeof globalThis.fetch = async (input, init) => { + const href = String(input); + const method = init?.method ?? "GET"; + requests.push(`${method} ${href}`); + const asked: Asked = { + url: href.split("?")[0]!, + href, + method, + headers: new Headers(init?.headers), + body: String(init?.body ?? ""), + }; + if (method === "GET" || method === "HEAD") return read(asked); + if (method === "PUT") return replace(asked); + if (method === "POST") return create(asked); + if (method === "DELETE") return remove(asked); + return answer(method === "PATCH" ? 415 : 405, null, {}, href); + }; + return { fetch, resources, requests }; +} From d827ebf6004719cb518309b20bc5ca0279a9348e Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Thu, 5 Nov 2026 21:13:20 +0100 Subject: [PATCH 05/19] feat(solid-memo-client): sign in to a Linked Web Storage storage An LWS storage takes no DPoP-bound Solid-OIDC token: it takes a Bearer token its authorization server issues for the OpenID Connect ID token, in an OAuth 2.0 Token Exchange. The Community Solid Server with LWS alone serves a DPoP request as no one's, so the app could read only what is public there, and write nothing. The client keeps the ID token in memory, and on a storage's LWS challenge checks what it names as LWS asks a client to, exchanges the ID token for that storage's token, and sends the request again with it. The ID token goes only to an authorization server on the identity provider's origin, as LWS binds it to none, and each storage's token only into that storage. The fork's challenge names its authorization server with as_uri, as an earlier draft had it; that is taken too. Inrupt's sign-in client gives the app no ID token, so this is the own client's alone. Refs: #594 --- docs/authentication.md | 13 + docs/solid-memo-client.md | 125 ++++- .../solid-memo-client/src/lwsAuth.test.ts | 517 ++++++++++++++++++ packages/solid-memo-client/src/lwsAuth.ts | 339 ++++++++++++ packages/solid-memo-client/src/oidcLogin.ts | 43 +- packages/solid-memo-client/src/oidcSession.ts | 191 +++++-- .../solid-memo-client/src/session.test.ts | 405 +++++++++++++- 7 files changed, 1572 insertions(+), 61 deletions(-) create mode 100644 packages/solid-memo-client/src/lwsAuth.test.ts create mode 100644 packages/solid-memo-client/src/lwsAuth.ts diff --git a/docs/authentication.md b/docs/authentication.md index be654a65e..91a24dd09 100644 --- a/docs/authentication.md +++ b/docs/authentication.md @@ -407,6 +407,13 @@ sent to such an address would name another server. The test is on the parsed address, so every spelling of a `//` at the start of the path is caught, and a path with one further along (`/a//b`) isn't refused here. +On `solid-memo-client`, a storage that speaks Linked Web Storage gets a +Bearer token the client exchanges the ID token for, sent within that +storage only. The client answers its challenge, and sends the request +again, before authFetch sees the response +([solid-memo-client.md](solid-memo-client.md#linked-web-storage)). +Inrupt's sign-in client gives the app no ID token short of ending the session, so it can't. + When a request fails and wasn't aborted, authFetch asks the same address with a HEAD as no one, the redirect not followed: @@ -439,6 +446,12 @@ session is gone. authFetch announces it `WWW-Authenticate`). The own storages are those the user's profile names, read as no one, and those entered on this page. +On `solid-memo-client`, a 401 from a Linked Web Storage storage reaches +authFetch only after the client tried to answer its challenge: a token +exchange, and the request sent again. A storage's token the storage +refuses is exchanged anew once first, so an expired one ends no +session. + A 401 from any other server, such as an instance the user registered in someone else's pod, says nothing of the session. The caller reads it as refused access, and the user stays logged in. A logout isn't an diff --git a/docs/solid-memo-client.md b/docs/solid-memo-client.md index 586d29a11..29248c6d4 100644 --- a/docs/solid-memo-client.md +++ b/docs/solid-memo-client.md @@ -131,9 +131,10 @@ not extractable, and binds the tokens to it. The browser keeps | IndexedDB, database `solid-memo-oidc`, store `session` | the session a reload restores: the issuer, the client, the redirect URL, the DPoP key pair as `CryptoKey` objects, the WebID, and the refresh token when the provider gave one | | IndexedDB, database `solid-memo-oidc`, store `registrations` | the clients providers registered the app as, by issuer and redirect URL; logout keeps them | | `sessionStorage`, `solid-memo:oidc-login` | during a login, the request under way: the issuer, the client, the redirect URL, the PKCE code verifier, the `state` and the `nonce`, and for a silent restore the page it left; removed when the provider sends the learner back | -| memory | the access token, and when it expires | +| memory | the access token and the ID token, and when each expires; the [Linked Web Storage](#linked-web-storage) storages the session met, each with its authorization server and its token, and when that expires | -The access token is never stored. IndexedDB keeps the key pair as +The access token, the ID token and the LWS tokens are never stored. A +logout, or a new login, drops them with the session they belong to. IndexedDB keeps the key pair as `CryptoKey` objects, which `localStorage`, holding only text, can't. ### Restore, refresh and logout @@ -172,7 +173,9 @@ The session's requests are in The access token is refreshed 30 seconds before it expires. Without a refresh token, the session ends once the access token expired, and the next request goes without one - ([authentication.md](authentication.md#an-expired-session)). + ([authentication.md](authentication.md#an-expired-session)). A + request to an LWS storage carries a Bearer token instead + ([below](#linked-web-storage)). - **A refresh, a login's save and the clear after a refused silent restore** hold the Web Lock `solid-memo:oidc-session`. A browser without Web Locks queues them within its tab. So two tabs never @@ -198,6 +201,115 @@ The session's requests are in session it read is still stored, in one IndexedDB transaction, so it can't bring back a session a logout cleared. +## Linked Web Storage + +A storage that speaks Linked Web Storage (LWS) authorization takes +neither the DPoP-bound access token nor a DPoP proof. It takes a +Bearer token its authorization server issues for the session's ID +token, in an OAuth 2.0 Token Exchange +([RFC 8693](https://www.rfc-editor.org/rfc/rfc8693)). The session gets +one in [lwsAuth.ts](../packages/solid-memo-client/src/lwsAuth.ts), +under its fetch in `oidcSession.ts`, so authFetch sees only the final +answer. The flow follows the +[LWS Protocol 1.0](https://www.w3.org/TR/lws10-core/) +working draft: + +1. **The challenge.** A request goes with the access token, as to any + Solid server. A 401 is an LWS challenge when its `WWW-Authenticate` + has a `Bearer` challenge with `resource_metadata` + ([RFC 9728](https://www.rfc-editor.org/rfc/rfc9728)), or with + `as_uri` ([below](#the-forks-challenge)). Its `Link` header names + the storage, with relation `https://www.w3.org/ns/lws#storage`. The + requested URL must be within that storage: on its origin, its path + the storage's or below it. +2. **The checks,** LWS's "Discovery Validation by a Client", before the + ID token goes anywhere. The `resource_metadata` URL has the + requested URL's scheme and authority, and its document names the + requested URL as its `resource`. Its first `authorization_servers` + entry is the authorization server. The storage's description, + asked for as `application/lws+cid`, is a `Storage` with a + `StorageRoot` service on the requested URL's scheme and authority. +3. **The authorization server's metadata** comes from its + `/.well-known/lws-configuration`, placed before the issuer's path as + [RFC 8414](https://www.rfc-editor.org/rfc/rfc8414) places a + well-known path. Its `issuer` is the one asked for, and its token + endpoint is `https:`, or `http:` to this machine, on the issuer's + origin. Where it lists its grant types and subject token types, the + lists include the token exchange and the ID token. +4. **The exchange** posts the token exchange grant with the storage as + `resource` and the ID token as `subject_token`, of type + `urn:ietf:params:oauth:token-type:id_token`. The client + authenticates as it does at login, and the request carries no DPoP + proof. The answer must be a `Bearer` token. +5. **The request again,** once, with `Authorization: Bearer` and no + `DPoP` header, keeping the caller's method, body, headers, redirect + mode and signal. A body the first try used up, a `Request`'s or a + stream, isn't sent again. The session still learns the storage, so the + next request to it goes with its token. + +A failed check, or an exchange the authorization server refuses, sends +nothing further: the caller gets the 401 as it came. The checks read +each document as no one, follow no redirect, and refuse one past 64 KiB, +since any server that answers 401 names them. A check or an +exchange that can't reach its server fails the request, as a dropped +connection does. The checks' requests are given up after 30 seconds, +or as soon as the caller's signal aborts. The exchange is given up +after 30 seconds. A caller whose signal aborts stops waiting for it, +and the other requests waiting for it still get its token. + +### The storage's token + +- **Kept** per storage, in memory, on the session: a logout or a new + login drops them all. A request to a URL within a storage met + before, the innermost one, goes with the storage's token first. +- **Renewed** 30 seconds before it expires, and one exchange at a time + per storage: the requests that wait share it. +- **Refused** with a 401: the session drops the token, exchanges once + more, and sends the request again once. A second 401 goes to the + caller. A token exchanged for this request isn't exchanged + again. +- **Not asked for again** with an ID token the authorization server + refused for that storage. The session exchanges again once it has + another ID token. +- **Sent only within its storage.** A Bearer token, unlike a DPoP one, + works for whoever holds it, and the app's fetch reaches other + servers than the user's storages. A request with it follows no + redirect: it fails when the caller asked for `redirect: "error"`, as + authFetch does, and otherwise comes back as the redirect it got. So + the token never goes to another origin, or outside the storage's + path on the same server. + +### The ID token + +The ID token comes with the login's tokens, and with a refresh's when +the provider sends a new one. A refresh that brings none keeps the one +the session has. When the ID token is about to expire, the session +refreshes first, until a refresh brings no ID token. With no ID token, +or an expired one, the session exchanges nothing and the 401 goes to +the caller. A reload restores the session with a refresh, so after a +reload the session has an ID token only from a provider that sends one +with refreshed tokens. + +### The trust rule + +The session sends the ID token only to an authorization server on the +origin of its own identity provider. The ID token's audience is the +app, so any LWS authorization server that trusts the identity provider +takes it as proof of the user. A storage that named an authorization +server of its own choosing would get a credential good at every +storage behind that provider. An LWS deployment whose authorization +server isn't on the identity provider's origin isn't supported yet: +its storages answer 401, as to no one. + +### The fork's challenge + +The Community Solid Server's LWS fork names its authorization server in +an `as_uri` parameter, and leaves out `resource_metadata`, which LWS +requires on every challenge. As a workaround, a `Bearer` challenge with +`as_uri` and no `resource_metadata` takes `as_uri` as the issuer, and +skips the resource metadata checks. The trust rule and the other +checks still hold. A challenge with both takes `resource_metadata`. + ## The public-read grant `grantPublicRead` @@ -247,6 +359,13 @@ Both pass the same contract. Beyond it, this client differs: keys. OpenID Connect allows this for a token the client receives from the token endpoint over TLS ([OpenID Connect Core 1.0, 3.1.3.7](https://openid.net/specs/openid-connect-core-1_0.html#IDTokenValidation)). +- **Linked Web Storage** works on this client only. The exchange needs + the ID token, which `@inrupt/solid-client-authn-browser` 5 gives the + app only in the address of an identity provider's logout, and from + no refresh. Inrupt's client sends a DPoP request to an LWS storage, + which reads it as no one's. Using an LWS storage needs a build + with `SOLID_CLIENT=solid-memo-client` + ([above](#linked-web-storage)). - **IndexedDB is required:** in a browser that refuses it, sign-in fails. Nothing falls back to a session that lasts only for the page. - **A dynamic registration is kept** until the site's data is cleared. diff --git a/packages/solid-memo-client/src/lwsAuth.test.ts b/packages/solid-memo-client/src/lwsAuth.test.ts new file mode 100644 index 000000000..22de23533 --- /dev/null +++ b/packages/solid-memo-client/src/lwsAuth.test.ts @@ -0,0 +1,517 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import * as oauth from "oauth4webapi"; +import { beforeEach, describe, expect, it, vi } from "vitest"; +import { LwsTokens, type LwsSession } from "./lwsAuth"; + +// What the Community Solid Server LWS fork (8.0.0-alpha.3) answers, as seen on it. +const ISSUER = "http://127.0.0.1:3989/"; +const STORAGE = `${ISSUER}alice/`; +const RESOURCE = new URL(`${STORAGE}notes/a.ttl`); +const TOKEN_ENDPOINT = `${ISSUER}.well-known/lws/token`; +const CONFIGURATION = `${ISSUER}.well-known/lws-configuration`; +const CHALLENGE = `Bearer as_uri="${ISSUER}", realm="${STORAGE}"`; +const STORAGE_LINK = `<${STORAGE}>; rel="https://www.w3.org/ns/lws#storage"`; + +const json = (body: unknown, status = 200) => + new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json" } }); + +/** The fork's metadata, as its /.well-known/lws-configuration serves it. */ +const configuration = (): Record => ({ + issuer: ISSUER, + token_endpoint: TOKEN_ENDPOINT, + jwks_uri: `${ISSUER}.well-known/lws/jwks`, + grant_types_supported: ["urn:ietf:params:oauth:grant-type:token-exchange"], + token_endpoint_auth_methods_supported: ["none"], + response_types_supported: ["token"], + claims_supported: ["sub", "iss", "client_id", "aud", "exp", "iat", "jti"], + subject_token_types_supported: [ + "urn:ietf:params:oauth:token-type:jwt", + "urn:ietf:params:oauth:token-type:id_token", + ], + subject_identifier_types_supported: ["https", "did:key"], +}); + +/** The fork's storage description (application/lws+cid). */ +const description = (): Record => ({ + "@context": ["https://www.w3.org/ns/cid/v1", "https://www.w3.org/ns/lws/v1"], + id: STORAGE, + type: "Storage", + service: [ + { type: "NotificationService", serviceEndpoint: "https://elsewhere.example/notify" }, + { type: "StorageRoot", serviceEndpoint: STORAGE }, + ], +}); + +/** The servers, by URL: what each answers. */ +let routes: Map Response | Promise>; +let requests: { url: string; init: RequestInit | undefined }[]; +let issued: number; + +function exchangeAnswer(): Response { + issued += 1; + return json({ + access_token: `lws-${issued}`, + issued_token_type: "urn:ietf:params:oauth:token-type:access_token", + token_type: "Bearer", + expires_in: 300, + }); +} + +const session: LwsSession = { + issuer: ISSUER, + client: () => ({ client_id: "client-1", token_endpoint_auth_method: "none" }), + idToken: vi.fn(async () => "id-token-1"), +}; + +beforeEach(() => { + issued = 0; + requests = []; + vi.mocked(session.idToken).mockClear(); + routes = new Map([ + [CONFIGURATION, () => json(configuration())], + [STORAGE, () => json(description())], + [TOKEN_ENDPOINT, exchangeAnswer], + ]); + vi.stubGlobal("fetch", async (input: RequestInfo | URL, init?: RequestInit) => { + const url = String(input); + requests.push({ url, init }); + const route = routes.get(url); + return route ? route(init) : new Response(null, { status: 404 }); + }); +}); + +/** The fork's 401 to `url`, with `challenge` and `link` as its headers. */ +function unauthorized(url = RESOURCE, challenge = CHALLENGE, link = STORAGE_LINK, status = 401) { + const headers = new Headers({ "www-authenticate": challenge }); + if (link !== "") headers.set("link", link); + const response = new Response(null, { status, headers }); + Object.defineProperty(response, "url", { value: url.href }); + return response; +} + +/** The response's challenges, as oauth4webapi parses them for the session's request (oidcSession.ts). */ +async function challengesOf(response: Response): Promise { + try { + await oauth.protectedResourceRequest("t", "GET", RESOURCE, undefined, null, { + [oauth.allowInsecureRequests]: true, + [oauth.customFetch]: async () => response, + }); + } catch (error) { + if (error instanceof oauth.WWWAuthenticateChallengeError) return error.cause; + } + throw new Error("No challenge"); +} + +async function discovered(tokens = new LwsTokens(session), response = unauthorized()) { + return tokens.discover(RESOURCE, response, await challengesOf(response)); +} + +describe("LwsTokens, on the fork's answers", () => { + it("checks the storage and its authorization server, then exchanges the ID token for its token", async () => { + const tokens = new LwsTokens(session); + const storage = await discovered(tokens); + expect(storage).toMatchObject({ + uri: STORAGE, + as: { issuer: ISSUER, token_endpoint: TOKEN_ENDPOINT }, + }); + expect( + requests.map(({ url, init }) => [url, new Headers(init?.headers).get("accept")]), + ).toEqual([ + [STORAGE, "application/lws+cid"], + [CONFIGURATION, "application/json"], + ]); + expect(requests.every(({ init }) => init?.credentials === "omit")).toBe(true); + expect(requests.every(({ init }) => init?.redirect === "manual")).toBe(true); + + await expect(tokens.token(storage!)).resolves.toEqual({ value: "lws-1", exchanged: true }); + const exchange = requests.at(-1)!; + expect(exchange.url).toBe(TOKEN_ENDPOINT); + expect(Object.fromEntries(new URLSearchParams(String(exchange.init?.body)))).toEqual({ + grant_type: "urn:ietf:params:oauth:grant-type:token-exchange", + resource: STORAGE, + subject_token: "id-token-1", + subject_token_type: "urn:ietf:params:oauth:token-type:id_token", + client_id: "client-1", + }); + const headers = new Headers(exchange.init?.headers); + expect(headers.get("dpop")).toBeNull(); + expect(headers.get("authorization")).toBeNull(); + + // Kept: the next request has it at once. + await expect(tokens.token(storage!)).resolves.toEqual({ value: "lws-1", exchanged: false }); + expect(issued).toBe(1); + }); + + it("knows a storage it checked by the URLs within it, the innermost first", async () => { + const tokens = new LwsTokens(session); + expect(tokens.storageOf(RESOURCE)).toBeUndefined(); + const storage = await discovered(tokens); + expect(tokens.storageOf(RESOURCE)).toBe(storage); + expect(tokens.storageOf(new URL(STORAGE))).toBe(storage); + expect(tokens.storageOf(new URL(`${ISSUER}alice`))).toBeUndefined(); + expect(tokens.storageOf(new URL(`${ISSUER}alicex/a`))).toBeUndefined(); + expect(tokens.storageOf(new URL("http://localhost:3989/alice/a"))).toBeUndefined(); + + // A storage inside another, as a server may have one. + const inner = `${STORAGE}notes`; + routes.set(inner, () => json(description())); + const innerStorage = await discovered( + tokens, + unauthorized(RESOURCE, CHALLENGE, `<${inner}>; rel="https://www.w3.org/ns/lws#storage"`), + ); + expect(innerStorage?.uri).toBe(inner); + expect(tokens.storageOf(RESOURCE)).toBe(innerStorage); + expect(tokens.storageOf(new URL(inner))).toBe(innerStorage); + expect(tokens.storageOf(new URL(`${STORAGE}other`))).toBe(storage); + + // Checked by two requests at once: one storage. + const twice = new LwsTokens(session); + const [one, other] = await Promise.all([discovered(twice), discovered(twice)]); + expect(one).toBe(other); + + // Met again: the storage as it was, with nothing asked. + requests = []; + await expect(discovered(tokens)).resolves.toBe(storage); + expect(requests).toEqual([]); + }); + + it("exchanges once for the requests that wait together, and again 30 seconds before the token expires", async () => { + const tokens = new LwsTokens(session); + const storage = (await discovered(tokens))!; + const [first, second] = await Promise.all([tokens.token(storage), tokens.token(storage)]); + expect(first).toEqual({ value: "lws-1", exchanged: true }); + expect(second).toEqual(first); + expect(issued).toBe(1); + vi.useFakeTimers({ toFake: ["Date"], now: Date.now() + 270_000 }); + try { + await expect(tokens.token(storage)).resolves.toEqual({ value: "lws-2", exchanged: true }); + } finally { + vi.useRealTimers(); + } + }); + + it("keeps a token whose lifetime the authorization server did not say", async () => { + routes.set(TOKEN_ENDPOINT, () => json({ access_token: "lws-forever", token_type: "Bearer" })); + const tokens = new LwsTokens(session); + const storage = (await discovered(tokens))!; + await tokens.token(storage); + vi.useFakeTimers({ toFake: ["Date"], now: Date.now() + 365 * 24 * 3_600_000 }); + try { + await expect(tokens.token(storage)).resolves.toEqual({ + value: "lws-forever", + exchanged: false, + }); + } finally { + vi.useRealTimers(); + } + }); + + it("forgets a refused token, but not a newer one", async () => { + const tokens = new LwsTokens(session); + const storage = (await discovered(tokens))!; + await tokens.token(storage); + tokens.forget(storage, "lws-0"); + await expect(tokens.token(storage)).resolves.toEqual({ value: "lws-1", exchanged: false }); + tokens.forget(storage, "lws-1"); + await expect(tokens.token(storage)).resolves.toEqual({ value: "lws-2", exchanged: true }); + }); + + it("exchanges nothing without an ID token", async () => { + vi.mocked(session.idToken).mockResolvedValueOnce(undefined); + const tokens = new LwsTokens(session); + const storage = (await discovered(tokens))!; + await expect(tokens.token(storage)).resolves.toBeUndefined(); + expect(requests.map(({ url }) => url)).not.toContain(TOKEN_ENDPOINT); + }); + + it.each([ + ["refuses the exchange", () => json({ error: "invalid_target" }, 400)], + ["gives a token of another type", () => json({ access_token: "x", token_type: "DPoP" })], + ])("has no token when the authorization server %s", async (_, answer) => { + routes.set(TOKEN_ENDPOINT, answer); + const tokens = new LwsTokens(session); + const storage = (await discovered(tokens))!; + await expect(tokens.token(storage)).resolves.toBeUndefined(); + expect(storage.token).toBeUndefined(); + }); + + it("sends an ID token the authorization server refused no more, until the session has another", async () => { + routes.set(TOKEN_ENDPOINT, () => json({ error: "invalid_grant" }, 400)); + const tokens = new LwsTokens(session); + const storage = (await discovered(tokens))!; + const exchanges = () => requests.filter(({ url }) => url === TOKEN_ENDPOINT).length; + await expect(tokens.token(storage)).resolves.toBeUndefined(); + await expect(tokens.token(storage)).resolves.toBeUndefined(); + expect(exchanges()).toBe(1); + vi.mocked(session.idToken).mockResolvedValueOnce("id-token-2"); + await expect(tokens.token(storage)).resolves.toBeUndefined(); + expect(exchanges()).toBe(2); + }); + + it("stops waiting for an exchange when the caller aborts, which goes on for the others", async () => { + let answer: () => void = () => {}; + routes.set( + TOKEN_ENDPOINT, + () => + new Promise((resolve) => { + answer = () => resolve(exchangeAnswer()); + }), + ); + const tokens = new LwsTokens(session); + const storage = (await discovered(tokens))!; + const caller = new AbortController(); + const waiting = tokens.token(storage, caller.signal); + const other = tokens.token(storage, new AbortController().signal); + await vi.waitFor(() => expect(requests.at(-1)?.url).toBe(TOKEN_ENDPOINT)); + caller.abort(new DOMException("Aborted", "AbortError")); + await expect(waiting).rejects.toMatchObject({ name: "AbortError" }); + answer(); + await expect(other).resolves.toEqual({ value: "lws-1", exchanged: true }); + + const aborted = AbortSignal.abort(new DOMException("Aborted", "AbortError")); + storage.token = undefined; + await expect(tokens.token(storage, aborted)).rejects.toMatchObject({ name: "AbortError" }); + }); + + it("gives the checks' requests the caller's signal, with a time limit", async () => { + const caller = new AbortController(); + const response = unauthorized(); + await new LwsTokens(session).discover( + RESOURCE, + response, + await challengesOf(response), + caller.signal, + ); + const signal = requests[0]?.init?.signal; + expect(signal?.aborted).toBe(false); + caller.abort(); + expect(signal?.aborted).toBe(true); + }); + + it("rejects when a check's request cannot be made", async () => { + routes.set(STORAGE, () => { + throw new TypeError("Failed to fetch"); + }); + await expect(discovered()).rejects.toThrow("Failed to fetch"); + }); +}); + +describe("LwsTokens, on LWS's resource metadata (RFC 9728)", () => { + const METADATA = `${ISSUER}.well-known/oauth-protected-resource/alice/notes/a.ttl`; + const specChallenge = (metadata = METADATA) => + `Bearer resource_metadata="${metadata}", error="invalid_token"`; + + beforeEach(() => { + routes.set(METADATA, () => json({ resource: RESOURCE.href, authorization_servers: [ISSUER] })); + }); + + it("takes the authorization server from the metadata that names the requested URI", async () => { + const response = unauthorized(RESOURCE, specChallenge()); + const storage = await new LwsTokens(session).discover( + new URL(`${RESOURCE.href}#it`), + response, + await challengesOf(response), + ); + expect(storage?.as.issuer).toBe(ISSUER); + expect(requests[0]?.url).toBe(METADATA); + }); + + it.each([ + ["is on another origin", "http://localhost:3989/meta", undefined], + ["isn't a URL", "/relative", undefined], + ["can't be read", `${ISSUER}missing`, undefined], + ["names another resource", METADATA, { resource: STORAGE, authorization_servers: [ISSUER] }], + ["names no authorization server", METADATA, { resource: RESOURCE.href }], + [ + "names one that isn't a string", + METADATA, + { resource: RESOURCE.href, authorization_servers: [1] }, + ], + ["isn't an object", METADATA, [RESOURCE.href]], + ["isn't JSON", METADATA, "not JSON"], + ])("refuses resource metadata that %s", async (_, metadata, body) => { + if (body !== undefined) { + routes.set(METADATA, () => (typeof body === "string" ? new Response(body) : json(body))); + } + const response = unauthorized(RESOURCE, specChallenge(metadata)); + await expect( + new LwsTokens(session).discover(RESOURCE, response, await challengesOf(response)), + ).resolves.toBeUndefined(); + expect(requests.map(({ url }) => url)).not.toContain(STORAGE); + }); +}); + +describe("LwsTokens, refusing", () => { + /** A JSON body past 64 KiB, its length said in advance or not. */ + const oversized = (said: boolean, document = description()) => { + const body = JSON.stringify({ ...document, padding: "x".repeat(64 * 1024) }); + return () => + said + ? new Response(body, { headers: { "content-length": String(body.length) } }) + : new Response( + new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode(body)); + controller.close(); + }, + }), + ); + }; + + it.each([ + ["says it is", true], + ["turns out to be", false], + ])("reads no storage description that %s past 64 KiB", async (_, said) => { + routes.set(STORAGE, oversized(said)); + await expect(discovered()).resolves.toBeUndefined(); + expect(requests.map(({ url }) => url)).toEqual([STORAGE]); + }); + + it("reads no resource metadata or authorization server metadata past 64 KiB", async () => { + const metadata = `${ISSUER}.well-known/oauth-protected-resource/alice/notes/a.ttl`; + routes.set( + metadata, + oversized(true, { resource: RESOURCE.href, authorization_servers: [ISSUER] }), + ); + const response = unauthorized(RESOURCE, `Bearer resource_metadata="${metadata}"`); + await expect( + new LwsTokens(session).discover(RESOURCE, response, await challengesOf(response)), + ).resolves.toBeUndefined(); + routes.set(CONFIGURATION, oversized(false, configuration())); + await expect(discovered()).resolves.toBeUndefined(); + }); + + it.each([ + ["no LWS challenge", unauthorized(RESOURCE, 'DPoP error="invalid_token"')], + [ + "a Bearer challenge that names no authorization server", + unauthorized(RESOURCE, 'Bearer realm="x"'), + ], + ["a status other than 401", unauthorized(RESOURCE, CHALLENGE, STORAGE_LINK, 403)], + ["no storage link", unauthorized(RESOURCE, CHALLENGE, "")], + [ + "a storage the URL isn't in", + unauthorized(RESOURCE, CHALLENGE, `<${ISSUER}bob/>; rel="https://www.w3.org/ns/lws#storage"`), + ], + [ + "a storage on another origin", + unauthorized( + RESOURCE, + CHALLENGE, + '; rel="https://www.w3.org/ns/lws#storage"', + ), + ], + [ + "a storage that only begins the same", + unauthorized( + new URL(`${ISSUER}alicex/a`), + CHALLENGE, + `<${ISSUER}alice>; rel="https://www.w3.org/ns/lws#storage"`, + ), + ], + ])("asks nothing on %s", async (_, response) => { + await expect( + new LwsTokens(session).discover( + new URL(response.url), + response, + await challengesOf(response), + ), + ).resolves.toBeUndefined(); + expect(requests).toEqual([]); + }); + + it.each([ + ["another origin", `Bearer as_uri="https://as.example/"`], + ["no URL", `Bearer as_uri="not a URL"`], + ])( + "sends nothing to an authorization server on %s than the identity provider's", + async (_, challenge) => { + await expect( + discovered(undefined, unauthorized(RESOURCE, challenge)), + ).resolves.toBeUndefined(); + expect(requests).toEqual([]); + }, + ); + + it.each([ + ["isn't a Storage", { ...description(), type: "Container" }], + ["has no services", { ...description(), service: undefined }], + [ + "has no StorageRoot", + { ...description(), service: [{ type: "Other", serviceEndpoint: STORAGE }] }, + ], + [ + "has a StorageRoot on another origin", + { + ...description(), + service: [{ type: "StorageRoot", serviceEndpoint: "https://x.example/" }], + }, + ], + [ + "has a StorageRoot without an endpoint", + { ...description(), service: [{ type: "StorageRoot" }] }, + ], + ["has a service that isn't an object", { ...description(), service: ["StorageRoot"] }], + ])("refuses a storage whose description %s", async (_, body) => { + routes.set(STORAGE, () => json(body)); + await expect(discovered()).resolves.toBeUndefined(); + expect(requests.map(({ url }) => url)).toEqual([STORAGE]); + }); + + it("takes a description whose types are lists, or full IRIs", async () => { + routes.set(STORAGE, () => + json({ + ...description(), + type: ["https://www.w3.org/ns/lws#Storage"], + service: [ + { type: ["https://www.w3.org/ns/lws#StorageRoot", "Other"], serviceEndpoint: STORAGE }, + ], + }), + ); + await expect(discovered()).resolves.toMatchObject({ uri: STORAGE }); + }); + + it.each([ + ["names another issuer", { issuer: "http://127.0.0.1:3989/other" }], + ["names no issuer", { issuer: undefined }], + ["has no token endpoint", { token_endpoint: undefined }], + ["has a token endpoint on another origin", { token_endpoint: "http://localhost:3989/token" }], + [ + "has a token endpoint over plain HTTP elsewhere", + { token_endpoint: "http://as.example/token" }, + ], + ["supports no token exchange", { grant_types_supported: ["client_credentials"] }], + [ + "takes no ID token", + { subject_token_types_supported: ["urn:ietf:params:oauth:token-type:jwt"] }, + ], + ])("refuses an authorization server whose metadata %s", async (_, change) => { + routes.set(CONFIGURATION, () => json({ ...configuration(), ...change })); + await expect(discovered()).resolves.toBeUndefined(); + }); + + it("takes metadata that leaves out what it supports, and an issuer without its path's slash", async () => { + routes.set(CONFIGURATION, () => + json({ + ...configuration(), + issuer: "http://127.0.0.1:3989", + grant_types_supported: undefined, + subject_token_types_supported: undefined, + }), + ); + await expect(discovered()).resolves.toMatchObject({ as: { issuer: "http://127.0.0.1:3989" } }); + }); + + it("finds the metadata of an issuer with a path where RFC 8414 puts it", async () => { + const issuer = `${ISSUER}tenant`; + const configured = `${ISSUER}.well-known/lws-configuration/tenant`; + routes.set(configured, () => json({ ...configuration(), issuer })); + await expect( + discovered(undefined, unauthorized(RESOURCE, `Bearer as_uri="${issuer}"`)), + ).resolves.toMatchObject({ as: { issuer } }); + expect(requests.map(({ url }) => url)).toContain(configured); + }); +}); diff --git a/packages/solid-memo-client/src/lwsAuth.ts b/packages/solid-memo-client/src/lwsAuth.ts new file mode 100644 index 000000000..ea2687200 --- /dev/null +++ b/packages/solid-memo-client/src/lwsAuth.ts @@ -0,0 +1,339 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import * as oauth from "oauth4webapi"; +import { type ReadLimits, ReadLimitError } from "@solid-memo/solid/client/interface"; +import { linkTargets } from "@solid-memo/solid/client/linkHeader"; +import { limitedFetch } from "@solid-memo/solid/client/readLimits"; +import { clientAuth, requestOptions, secureEndpoint, timeLimited } from "./oidcLogin"; + +/** + * Linked Web Storage (LWS) authorization for a session that logged in + * with Solid-OIDC (docs/solid-memo-client.md#linked-web-storage): a + * storage that answers 401 with an LWS challenge is checked as LWS + * Protocol 1.0's "Discovery Validation by a Client" says, then the + * session's ID token is exchanged (RFC 8693) at the storage's + * authorization server for a Bearer token whose audience is the storage. + * The tokens are kept per storage, in memory, for the session's life; + * oidcSession.ts sends them. + * + * Two rules of the app's own, both because a Bearer token, unlike a DPoP + * one, works for whoever holds it: + * + * - The ID token goes only to an authorization server on the origin of + * the session's own identity provider. Its audience is the app, so any + * authorization server that trusts that provider would take it; a + * server that names its own authorization server never gets it. + * - A storage's token goes only to URLs within that storage (same origin, + * and the storage's path as a prefix), never elsewhere on its server. + * + * Deviation, for the Community Solid Server LWS fork: its challenge + * names the authorization server in `as_uri` and has no + * `resource_metadata` (RFC 9728), which LWS requires. A challenge with + * `as_uri` alone takes it as the authorization server's issuer; the + * trust rule above holds it to the identity provider's origin. + */ + +/** The Link relation naming the storage a resource is in. */ +const LWS_STORAGE = "https://www.w3.org/ns/lws#storage"; + +const TOKEN_EXCHANGE = "urn:ietf:params:oauth:grant-type:token-exchange"; +const ID_TOKEN_TYPE = "urn:ietf:params:oauth:token-type:id_token"; + +/** How long before it expires a storage's token is exchanged anew, in ms. */ +const RENEW_MARGIN = 30_000; + +/** A storage the app talks LWS with: where it is, its authorization server, and its token. */ +export interface LwsStorage { + readonly uri: string; + readonly as: oauth.AuthorizationServer; + token?: { value: string; expiresAt: number }; + exchanging?: Promise; + /** The ID token the authorization server refused for this storage: not sent again. */ + refused?: string; +} + +/** What LWS authorization needs of the session: its provider, client and ID token. */ +export interface LwsSession { + /** The issuer of the session's identity provider. */ + readonly issuer: string; + client(): oauth.Client; + /** The ID token, refreshed when it is about to expire; none when there is none to have. */ + idToken(): Promise; +} + +type Json = Record; + +function isObject(value: unknown): value is Json { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +/** Whether a `type` (a string, or a list of strings) names one of `names`. */ +function names(type: unknown, ...wanted: string[]): boolean { + const types = Array.isArray(type) ? type : [type]; + return types.some((one) => wanted.includes(one)); +} + +/** Whether `url` is within `storage`: on its origin, and its path the storage's or below it. */ +function within(url: URL, storage: string): boolean { + const root = new URL(storage); + const folder = root.pathname.endsWith("/") ? root.pathname : `${root.pathname}/`; + return ( + root.origin === url.origin && + (url.pathname === root.pathname || url.pathname.startsWith(folder)) + ); +} + +/** The request's own URI: without its fragment, which is never sent. */ +function requested(url: URL): string { + const uri = new URL(url); + uri.hash = ""; + return uri.href; +} + +/** + * How much of a discovery document is read: any server that answers a + * session's request with a 401 names them, so a larger one is refused + * rather than read into the tab's memory. + */ +const DISCOVERY_LIMITS: ReadLimits = { bytes: 64 * 1024, quads: 0, characters: 64 * 1024 }; + +/** + * A JSON object read with a GET, as no one, following no redirect, and + * no larger than DISCOVERY_LIMITS; none for anything else. + */ +async function getJson( + url: string, + accept: string, + signal: AbortSignal | null | undefined, +): Promise { + let response: Response; + try { + response = await limitedFetch(globalThis.fetch, DISCOVERY_LIMITS)(url, { + headers: { accept }, + credentials: "omit", + redirect: "manual", + signal: timeLimited(signal), + }); + } catch (error) { + // Its Content-Length says it is too large. + if (error instanceof ReadLimitError) return undefined; + throw error; + } + if (!response.ok) return undefined; + const body: unknown = await response + .text() + .then(JSON.parse) + .catch(() => undefined); + return isObject(body) ? body : undefined; +} + +/** Waits for `work`, or rejects as soon as `signal` aborts; `work` goes on for whoever else waits for it. */ +function abortable(work: Promise, signal: AbortSignal | null | undefined): Promise { + if (!signal) return work; + return new Promise((resolve, reject) => { + const abort = () => reject(signal.reason); + if (signal.aborted) abort(); + signal.addEventListener("abort", abort, { once: true }); + work.then(resolve, reject).finally(() => signal.removeEventListener("abort", abort)); + }); +} + +/** + * The authorization server's issuer the challenge leads to. With + * `resource_metadata` (LWS): the metadata is on the requested URI's + * scheme and authority, names that URI as its `resource`, and its first + * authorization server is the one. Without it, the fork's `as_uri`. + */ +async function issuerOf( + url: URL, + challenge: oauth.WWWAuthenticateChallenge, + signal: AbortSignal | null | undefined, +): Promise { + const { resource_metadata: metadataUri, as_uri: asUri } = challenge.parameters; + if (metadataUri === undefined) return asUri; + const metadataUrl = URL.parse(metadataUri); + if (metadataUrl?.origin !== url.origin) return undefined; + const metadata = await getJson(metadataUrl.href, "application/json", signal); + if (metadata?.resource !== requested(url)) return undefined; + const servers = metadata.authorization_servers; + return Array.isArray(servers) && typeof servers[0] === "string" ? servers[0] : undefined; +} + +/** + * Whether the storage's description (application/lws+cid) is a + * Storage's, with a StorageRoot service on the requested URI's scheme + * and authority. + */ +async function describesStorage( + storage: string, + url: URL, + signal: AbortSignal | null | undefined, +): Promise { + const description = await getJson(storage, "application/lws+cid", signal); + if (!names(description?.type, "Storage", "https://www.w3.org/ns/lws#Storage")) return false; + const services: unknown[] = Array.isArray(description?.service) ? description.service : []; + return services.some( + (service) => + isObject(service) && + names(service.type, "StorageRoot", "https://www.w3.org/ns/lws#StorageRoot") && + typeof service.serviceEndpoint === "string" && + URL.parse(service.serviceEndpoint)?.origin === url.origin, + ); +} + +/** Whether a metadata list the server may leave out lists `value` when it is there. */ +const listsOrOmits = (list: unknown, value: string) => + list === undefined || (Array.isArray(list) && list.includes(value)); + +/** + * The authorization server's metadata, from its + * /.well-known/lws-configuration, placed as RFC 8414 places a + * well-known path: before the issuer's path. Its issuer must be the one + * asked for, its token endpoint `https:` (or `http:` on this machine) on + * the issuer's origin, and the token exchange of ID tokens supported + * where it lists what it supports. + */ +async function lwsConfiguration( + issuer: URL, + signal: AbortSignal | null | undefined, +): Promise { + const path = issuer.pathname === "/" ? "" : issuer.pathname; + const metadata = await getJson( + `${issuer.origin}/.well-known/lws-configuration${path}`, + "application/json", + signal, + ); + const tokenEndpoint = metadata?.token_endpoint; + if ( + typeof metadata?.issuer !== "string" || + URL.parse(metadata.issuer)?.href !== issuer.href || + typeof tokenEndpoint !== "string" || + secureEndpoint(tokenEndpoint)?.origin !== issuer.origin || + !listsOrOmits(metadata.grant_types_supported, TOKEN_EXCHANGE) || + !listsOrOmits(metadata.subject_token_types_supported, ID_TOKEN_TYPE) + ) { + return undefined; + } + return { issuer: metadata.issuer, token_endpoint: tokenEndpoint }; +} + +/** The tokens of the LWS storages a session has met, kept on the session (oidcSession.ts). */ +export class LwsTokens { + readonly #session: LwsSession; + readonly #storages = new Map(); + + constructor(session: LwsSession) { + this.#session = session; + } + + /** The storage `url` is within, of those met, the innermost; none when it is in none. */ + storageOf(url: URL): LwsStorage | undefined { + let found: LwsStorage | undefined; + for (const storage of this.#storages.values()) { + if (within(url, storage.uri) && storage.uri.length > (found?.uri.length ?? -1)) { + found = storage; + } + } + return found; + } + + /** + * The LWS storage a 401 to `url` names, once checked; none when the + * answer is no LWS challenge, or a check fails. A check's request that + * cannot be made rejects, as the request would have. + */ + async discover( + url: URL, + response: Response, + challenges: oauth.WWWAuthenticateChallenge[], + signal?: AbortSignal | null, + ): Promise { + const challenge = challenges.find( + ({ scheme, parameters }) => + scheme === "bearer" && + (parameters.resource_metadata !== undefined || parameters.as_uri !== undefined), + ); + if (response.status !== 401 || challenge === undefined) return undefined; + const uri = linkTargets(response, LWS_STORAGE, url.href).find((target) => within(url, target)); + if (uri === undefined) return undefined; + const known = this.#storages.get(uri); + if (known !== undefined) return known; + const issuer = URL.parse((await issuerOf(url, challenge, signal)) ?? ""); + if ( + issuer?.origin !== new URL(this.#session.issuer).origin || + !(await describesStorage(uri, url, signal)) + ) { + return undefined; + } + const as = await lwsConfiguration(issuer, signal); + if (as === undefined) return undefined; + // Another request may have checked it meanwhile: one storage, one token at a time. + const storage = this.#storages.get(uri) ?? { uri, as }; + this.#storages.set(uri, storage); + return storage; + } + + /** + * The storage's token: the one it has until 30 seconds before it + * expires, else a new one, from one exchange at a time per storage. + * `exchanged` when it was exchanged for this call. None when the + * session has no ID token, or the authorization server refuses it: + * that ID token is not sent for the storage again. + */ + async token( + storage: LwsStorage, + signal?: AbortSignal | null, + ): Promise<{ value: string; exchanged: boolean } | undefined> { + const kept = storage.token; + if (kept !== undefined && Date.now() < kept.expiresAt - RENEW_MARGIN) { + return { value: kept.value, exchanged: false }; + } + storage.exchanging ??= this.#exchange(storage).finally(() => { + storage.exchanging = undefined; + }); + const value = await abortable(storage.exchanging, signal); + return value === undefined ? undefined : { value, exchanged: true }; + } + + /** Forgets the storage's token `value`, which it refused; a newer one is kept. */ + forget(storage: LwsStorage, value: string): void { + if (storage.token?.value === value) storage.token = undefined; + } + + /** + * Exchanges the session's ID token for one of the storage's (RFC 8693): + * its `resource` the storage, the client authenticating as at login, + * and no DPoP, as LWS tokens are Bearer tokens. + */ + async #exchange(storage: LwsStorage): Promise { + const idToken = await this.#session.idToken(); + if (idToken === undefined || idToken === storage.refused) return undefined; + const client = this.#session.client(); + const response = await oauth.genericTokenEndpointRequest( + storage.as, + client, + clientAuth(client), + TOKEN_EXCHANGE, + { resource: storage.uri, subject_token: idToken, subject_token_type: ID_TOKEN_TYPE }, + // The token endpoint was checked: https:, or http: on this machine. + requestOptions(storage.as.token_endpoint!), + ); + const tokens = await oauth + .processGenericTokenEndpointResponse(storage.as, client, response) + .catch(() => undefined); + if (tokens?.token_type !== "bearer") { + storage.refused = idToken; + return undefined; + } + storage.token = { + value: tokens.access_token, + expiresAt: + tokens.expires_in === undefined + ? Number.POSITIVE_INFINITY + : Date.now() + tokens.expires_in * 1000, + }; + return tokens.access_token; + } +} diff --git a/packages/solid-memo-client/src/oidcLogin.ts b/packages/solid-memo-client/src/oidcLogin.ts index c6c38d72e..8b1a02a47 100644 --- a/packages/solid-memo-client/src/oidcLogin.ts +++ b/packages/solid-memo-client/src/oidcLogin.ts @@ -33,12 +33,16 @@ export interface PendingLogin { silentFrom?: string; } -/** The tokens of a session: the access token, in memory only. */ +/** The tokens of a session: the access token and the ID token, in memory only. */ export interface Tokens { accessToken: string; /** When the access token expires, in ms since the epoch; Infinity when the provider did not say. */ expiresAt: number; refreshToken?: string; + /** The ID token, which a Linked Web Storage authorization server takes for its own tokens (lwsAuth.ts). */ + idToken?: string; + /** When the ID token expires, in ms since the epoch (its exp claim). */ + idTokenExpiresAt?: number; } /** The parameters a provider sends back on the redirect URL, removed once read. */ @@ -60,28 +64,38 @@ export function isLoopbackHttp(url: URL): boolean { /** How long a request to the provider may take, in ms, before it is given up. */ const REQUEST_TIMEOUT = 30_000; +/** A signal that aborts after the time limit, or as soon as `signal`, the caller's, does. */ +export function timeLimited(signal?: AbortSignal | null): AbortSignal { + const timeout = AbortSignal.timeout(REQUEST_TIMEOUT); + return signal ? AbortSignal.any([signal, timeout]) : timeout; +} + /** * The options of a request to the provider: http: only to this machine, * and a time limit, so that a provider that does not answer cannot keep * a login, restore or refresh waiting, and the Web Lock a refresh holds * (oidcSession.ts), forever. */ -function requestOptions(url: string) { +export function requestOptions(url: string) { return { [oauth.allowInsecureRequests]: isLoopbackHttp(new URL(url)), - signal: () => AbortSignal.timeout(REQUEST_TIMEOUT), + signal: () => timeLimited(), }; } +/** The endpoint, when it is a URL the app sends requests to: `https:`, or `http:` to this machine. */ +export function secureEndpoint(endpoint: string): URL | undefined { + const url = URL.parse(endpoint); + return url !== null && (url.protocol === "https:" || isLoopbackHttp(url)) ? url : undefined; +} + /** * Refuses an endpoint the provider's metadata names that is not * `https:`, or `http:` to this machine: the browser is sent to the * authorization endpoint, and the app's requests go to the others. */ function checkEndpoint(issuer: string, name: string, endpoint: string | undefined): void { - if (endpoint === undefined) return; - const url = URL.parse(endpoint); - if (url === null || !(url.protocol === "https:" || isLoopbackHttp(url))) { + if (endpoint !== undefined && secureEndpoint(endpoint) === undefined) { throw new Error(`${issuer}'s ${name} is neither https: nor on this machine`); } } @@ -113,7 +127,7 @@ export async function discover(issuer: string): Promise(request: () => Promise): Promise { } } +/** + * The tokens to keep from the token endpoint's answer, the ID token with + * them when it sent one, which oauth4webapi checked (issuer, audience, + * expiry). The ID token stays in memory, as the access token does. + */ function tokensOf(response: oauth.TokenEndpointResponse): Tokens { + const claims = oauth.getValidatedIdTokenClaims(response); return { accessToken: response.access_token, expiresAt: @@ -262,6 +282,10 @@ function tokensOf(response: oauth.TokenEndpointResponse): Tokens { ? Number.POSITIVE_INFINITY : Date.now() + response.expires_in * 1000, ...(response.refresh_token !== undefined && { refreshToken: response.refresh_token }), + ...(claims !== undefined && { + idToken: response.id_token, + idTokenExpiresAt: claims.exp * 1000, + }), }; } @@ -315,7 +339,10 @@ export async function completeLogin( return { tokens: tokensOf(response), webId: webIdOf(claims) }; } -/** New tokens for `refreshToken`, bound to `dpop`'s key as the old ones were. */ +/** + * New tokens for `refreshToken`, bound to `dpop`'s key as the old ones + * were, with a new ID token when the provider sends one. + */ export async function refreshTokens( as: oauth.AuthorizationServer, client: oauth.Client, diff --git a/packages/solid-memo-client/src/oidcSession.ts b/packages/solid-memo-client/src/oidcSession.ts index 5e80b234b..7ff618dc3 100644 --- a/packages/solid-memo-client/src/oidcSession.ts +++ b/packages/solid-memo-client/src/oidcSession.ts @@ -3,17 +3,20 @@ // SPDX-License-Identifier: MIT import * as oauth from "oauth4webapi"; +import { LwsTokens, type LwsStorage } from "./lwsAuth"; import { isLoopbackHttp, isRefusal, refreshTokens, type Tokens } from "./oidcLogin"; import { clearSession, loadSession, replaceSession, type StoredSession } from "./oidcStore"; /** - * A logged-in session in this tab: its tokens, the access token in - * memory only, and the fetch that sends them, DPoP-bound, refreshing the - * access token as it is about to expire. The tab has one current - * session, as the app has one user (currentSession). + * A logged-in session in this tab: its tokens, the access and ID tokens + * in memory only, and the fetch that sends them, DPoP-bound, refreshing + * the access token as it is about to expire. On a Linked Web Storage + * storage it sends the Bearer token it exchanges the ID token for + * (lwsAuth.ts). The tab has one current session, as the app has one user + * (currentSession). */ -/** How long before it expires an access token is refreshed, in ms. */ +/** How long before it expires an access or ID token is refreshed, in ms. */ const REFRESH_MARGIN = 30_000; /** The last write of the stored session in this tab, when the browser has no Web Locks. */ @@ -101,13 +104,66 @@ function withProof( return { ...init, headers }; } +/** A request of the caller's, as the session sends it, once or again. */ +interface Sent { + input: RequestInfo | URL; + init: RequestInit | undefined; + url: URL; + method: string; +} + +/** + * The redirect mode of a request with an LWS storage's Bearer token: no + * redirect followed, as the browser would send the token on to any + * address on the same origin, outside the storage too. "error" when the + * caller asked for it (authFetch.ts), so a redirect fails as it expects; + * "manual" otherwise, the redirect coming back as it is. + */ +function bearerRedirect({ input, init }: Sent): RequestRedirect { + const asked = init?.redirect ?? (input instanceof Request ? input.redirect : undefined); + return asked === "error" ? "error" : "manual"; +} + +/** + * Sends the request with `token`, Bearer, or DPoP with `dpop`'s proof. + * A Bearer request follows no redirect (bearerRedirect). A challenge + * comes back as the response it came with, its parsed challenges beside + * it. + */ +async function send( + request: Sent, + token: string, + dpop?: oauth.DPoPHandle, +): Promise<{ response: Response; challenge?: oauth.WWWAuthenticateChallengeError }> { + const { input, init, url, method } = request; + const redirect = dpop === undefined ? { redirect: bearerRedirect(request) } : {}; + try { + const response = await oauth.protectedResourceRequest(token, method, url, undefined, null, { + ...(dpop !== undefined && { DPoP: dpop }), + [oauth.allowInsecureRequests]: url.protocol === "http:", + // oauth4webapi builds the Authorization header and proof; the request itself is the caller's. + [oauth.customFetch]: (_, built) => + globalThis.fetch(input, { ...withProof(input, init, built.headers), ...redirect }), + }); + return { response }; + } catch (error) { + if (!(error instanceof oauth.WWWAuthenticateChallengeError)) throw error; + return { response: error.response, challenge: error }; + } +} + export class OidcSession { readonly webId: string; readonly #as: oauth.AuthorizationServer; + #client: oauth.Client; #dpop: oauth.DPoPHandle; readonly #refreshable: boolean; #tokens: Tokens; #refreshing: Promise | undefined; + /** Whether a refresh may bring an ID token: until one brought none. */ + #refreshGivesIdToken = true; + /** The LWS storages met, and their tokens: this session's alone, gone with it. */ + readonly #lws: LwsTokens; constructor( as: oauth.AuthorizationServer, @@ -117,15 +173,26 @@ export class OidcSession { this.webId = stored.webId; this.#refreshable = stored.refreshToken !== undefined; this.#as = as; + this.#client = stored.client; this.#dpop = oauth.DPoP(stored.client, stored.keys); this.#tokens = tokens; + this.#lws = new LwsTokens({ + issuer: as.issuer, + client: () => this.#client, + idToken: () => this.#idToken(), + }); } /** - * Fetches as the session's user: Authorization: DPoP with the access - * token and a proof for this request (its method, and its URL without - * query or fragment). A resource server's 401 comes back as it is, - * once more after a DPoP nonce it asked for. + * Fetches as the session's user. Within a Linked Web Storage storage + * met before: Authorization: Bearer with its token (lwsAuth.ts). + * Elsewhere: Authorization: DPoP with the access token and a proof for + * this request (its method, and its URL without query or fragment), + * once more after a DPoP nonce the server asked for. A 401 with an + * LWS challenge is sent once more with the storage's token, when its + * checks pass and its body can be sent again. A 401 to a storage's + * token that was not new is sent once more with a new one. Any other + * 401 comes back as it is. */ readonly fetch: typeof globalThis.fetch = async (input, init) => { const url = new URL(input instanceof Request ? input.url : String(input), window.location.href); @@ -137,24 +204,52 @@ export class OidcSession { const method = ( init?.method ?? (input instanceof Request ? input.method : "GET") ).toUpperCase(); - const send = async (again: boolean): Promise => { - try { - return await oauth.protectedResourceRequest(accessToken, method, url, undefined, null, { - DPoP: this.#dpop, - [oauth.allowInsecureRequests]: url.protocol === "http:", - // oauth4webapi builds the proof; the request itself is the caller's. - [oauth.customFetch]: (_, built) => - globalThis.fetch(input, withProof(input, init, built.headers)), - }); - } catch (error) { - if (!(error instanceof oauth.WWWAuthenticateChallengeError)) throw error; - if (again && oauth.isDPoPNonceError(error) && replayable(input, init)) return send(false); - return error.response; - } - }; - return send(true); + const request: Sent = { input, init, url, method }; + const storage = this.#lws.storageOf(url); + const asLws = storage && (await this.#withLws(storage, request)); + return asLws ?? this.#withDpop(accessToken, request, storage); }; + /** + * The request with the access token, DPoP-bound, and what an LWS + * challenge to it leads to; not for `tried`, the storage this request + * already found no token for. + */ + async #withDpop(accessToken: string, request: Sent, tried?: LwsStorage): Promise { + let { response, challenge } = await send(request, accessToken, this.#dpop); + const again = replayable(request.input, request.init); + if (challenge !== undefined && again && oauth.isDPoPNonceError(challenge)) { + ({ response, challenge } = await send(request, accessToken, this.#dpop)); + } + if (challenge === undefined) return response; + const storage = await this.#lws.discover( + request.url, + response, + challenge.cause, + signalOf(request), + ); + if (storage === undefined || storage === tried || !again) return response; + return (await this.#withLws(storage, request)) ?? response; + } + + /** + * The request with the LWS storage's token, and once more with a new + * one when the storage refuses one that was not new; none when the + * session has no token for the storage. + */ + async #withLws(storage: LwsStorage, request: Sent): Promise { + const signal = signalOf(request); + const token = await this.#lws.token(storage, signal); + if (token === undefined) return undefined; + const { response } = await send(request, token.value); + if (response.status !== 401 || token.exchanged || !replayable(request.input, request.init)) { + return response; + } + this.#lws.forget(storage, token.value); + const renewed = await this.#lws.token(storage, signal); + return renewed === undefined ? response : (await send(request, renewed.value)).response; + } + /** * The access token, refreshed first when it is about to expire; none * once the provider refused to, or once it expired with no refresh @@ -162,32 +257,58 @@ export class OidcSession { * with no session as one that expired). */ async #accessToken(): Promise { - if (Date.now() >= this.#tokens.expiresAt - REFRESH_MARGIN) { - this.#refreshing ??= this.#refresh().finally(() => { - this.#refreshing = undefined; - }); - await this.#refreshing; - } + if (Date.now() >= this.#tokens.expiresAt - REFRESH_MARGIN) await this.#refreshOnce(); if (currentSession() === this && Date.now() >= this.#tokens.expiresAt) setCurrentSession(null); return currentSession() === this ? this.#tokens.accessToken : undefined; } + /** + * The ID token, for an LWS token exchange: refreshed first when it is + * about to expire and a refresh may bring a new one; none once it + * expired, or the session ended. + */ + async #idToken(): Promise { + const expiresAt = this.#tokens.idTokenExpiresAt ?? 0; + if (Date.now() >= expiresAt - REFRESH_MARGIN && this.#refreshGivesIdToken) { + await this.#refreshOnce(); + } + const { idToken, idTokenExpiresAt = 0 } = this.#tokens; + return currentSession() === this && Date.now() < idTokenExpiresAt ? idToken : undefined; + } + + /** Refreshes the tokens, one refresh at a time however many requests wait for it. */ + async #refreshOnce(): Promise { + this.#refreshing ??= this.#refresh().finally(() => { + this.#refreshing = undefined; + }); + await this.#refreshing; + } + /** * Swaps in fresh tokens, and the key pair they are bound to: the stored - * session's, which a login in another tab may have replaced. Without a - * refresh token the one it has goes on until it expires; when there is - * none to refresh with any more, the session ends here. + * session's, which a login in another tab may have replaced. The ID + * token is kept when the provider sent no new one. Without a refresh + * token the one it has goes on until it expires; when there is none to + * refresh with any more, the session ends here. */ async #refresh(): Promise { if (!this.#refreshable) return; const refreshed = await refreshStored(this.#as, this.webId); if (typeof refreshed === "object") { + const { idToken, idTokenExpiresAt } = this.#tokens; + this.#client = refreshed.stored.client; this.#dpop = oauth.DPoP(refreshed.stored.client, refreshed.stored.keys); - this.#tokens = refreshed.tokens; + this.#refreshGivesIdToken = refreshed.tokens.idToken !== undefined; + this.#tokens = { idToken, idTokenExpiresAt, ...refreshed.tokens }; } else if (currentSession() === this) setCurrentSession(null); } } +/** The caller's signal, from its init or its Request. */ +function signalOf({ input, init }: Sent): AbortSignal | null | undefined { + return init?.signal ?? (input instanceof Request ? input.signal : undefined); +} + let current: OidcSession | null = null; /** The tab's session; none when logged out. */ diff --git a/packages/solid-memo-client/src/session.test.ts b/packages/solid-memo-client/src/session.test.ts index a8bc90602..c2751ad64 100644 --- a/packages/solid-memo-client/src/session.test.ts +++ b/packages/solid-memo-client/src/session.test.ts @@ -124,8 +124,11 @@ interface Provider { /** Whether it gives refresh tokens, and a new one on every refresh. */ refreshTokensIssued: boolean; rotates: boolean; - /** Whether it sends an ID token with the authorization code grant's tokens. */ + /** Whether it sends an ID token with the authorization code grant's tokens, and with a refresh's. */ idToken: boolean; + refreshIdToken: boolean; + /** The ID tokens it issued, last one last. */ + idTokens: string[]; failure?: Error; /** What the token endpoint fails with, unreachable. */ tokenFailure?: Error; @@ -177,9 +180,12 @@ async function tokenEndpoint(init: RequestInit | undefined): Promise { await provider.refreshGate; const used = parameters.get("refresh_token") ?? ""; if (!provider.refreshTokens.has(used)) return json({ error: "invalid_grant" }, 400); - if (!provider.rotates) return json({ ...tokens, refresh_token: undefined }); + const refreshed = provider.refreshIdToken + ? { ...tokens, id_token: await idToken({ aud: parameters.get("client_id") }) } + : tokens; + if (!provider.rotates) return json({ ...refreshed, refresh_token: undefined }); provider.refreshTokens.delete(used); - return json(tokens); + return json(refreshed); } const grant = provider.grants.get(parameters.get("code") ?? ""); if (grant === undefined) return json({ error: "invalid_grant" }, 400); @@ -191,20 +197,97 @@ async function tokenEndpoint(init: RequestInit | undefined): Promise { ) { return json({ error: "invalid_grant" }, 400); } + return json( + provider.idToken + ? { ...tokens, id_token: await idToken({ aud: grant.clientId, nonce: grant.nonce }) } + : tokens, + ); +} + +/** An ID token the provider issues, valid for 300 s, with `claims` besides its own. */ +async function idToken(claims: Record): Promise { const now = Math.floor(Date.now() / 1000); - const claims = Object.fromEntries( - Object.entries({ - iss: ISSUER, - aud: grant.clientId, - sub: provider.webId, - webid: provider.webId, - nonce: grant.nonce, - iat: now, - exp: now + 300, - ...provider.claims, - }).filter(([, value]) => value !== undefined), + const token = await signedJwt( + Object.fromEntries( + Object.entries({ + iss: ISSUER, + sub: provider.webId, + webid: provider.webId, + iat: now, + exp: now + 300, + ...claims, + ...provider.claims, + }).filter(([, value]) => value !== undefined), + ), ); - return json(provider.idToken ? { ...tokens, id_token: await signedJwt(claims) } : tokens); + provider.idTokens.push(token); + return token; +} + +/** A Linked Web Storage storage, its authorization server the identity provider's (lwsAuth.ts). */ +const STORAGE = "https://pod.example/alice/"; +const LWS_TOKEN_ENDPOINT = `${ISSUER}.well-known/lws/token`; + +/** The LWS authorization server on the provider's origin, as the Community Solid Server LWS fork runs one. */ +interface Lws { + configuration: Record; + exchanges: { parameters: URLSearchParams; dpop: string | null }[]; + /** The storage's tokens it still takes. */ + valid: Set; + /** Whether the storage refuses every token. */ + refusesAll: boolean; + issued: number; +} + +let lws: Lws; + +function exchangeEndpoint(init: RequestInit | undefined): Response { + const parameters = new URLSearchParams(String(init?.body)); + lws.exchanges.push({ parameters, dpop: new Headers(init?.headers).get("dpop") }); + if (!provider.idTokens.includes(parameters.get("subject_token") ?? "")) { + return json({ error: "invalid_grant" }, 400); + } + lws.issued += 1; + const token = `lws-${lws.issued}`; + lws.valid.add(token); + return json({ + access_token: token, + issued_token_type: "urn:ietf:params:oauth:token-type:access_token", + token_type: "Bearer", + expires_in: 300, + }); +} + +/** + * The storage as the fork serves it: its description, a request with + * one of its tokens within it, and a 401 with its challenge to anything + * else, a DPoP request among them, which it reads as no one's. + */ +function lwsPod(asUri = ISSUER) { + return vi.fn(async (input, init) => { + const url = input instanceof Request ? input.url : String(input); + const headers = new Headers(init?.headers); + if (url === STORAGE && headers.get("accept") === "application/lws+cid") { + return json({ + "@context": ["https://www.w3.org/ns/cid/v1", "https://www.w3.org/ns/lws/v1"], + id: STORAGE, + type: "Storage", + service: [{ type: "StorageRoot", serviceEndpoint: STORAGE }], + }); + } + const [scheme, token] = (headers.get("authorization") ?? "").split(" "); + const bearer = scheme === "Bearer"; + if (bearer && !lws.refusesAll && lws.valid.has(token ?? "") && url.startsWith(STORAGE)) { + return new Response("ok"); + } + return new Response(null, { + status: 401, + headers: { + "www-authenticate": `Bearer as_uri="${asUri}", realm="${STORAGE}"${bearer ? ', error="invalid_token"' : ""}`, + link: `<${STORAGE}>; rel="https://www.w3.org/ns/lws#storage"`, + }, + }); + }); } const fakeFetch: typeof globalThis.fetch = async (input, init) => { @@ -227,6 +310,8 @@ const fakeFetch: typeof globalThis.fetch = async (input, init) => { return json({ ...metadata, client_id: id, ...secret }, 201); } if (url === `${ISSUER}token`) return tokenEndpoint(init); + if (url === `${ISSUER}.well-known/lws-configuration`) return json(lws.configuration); + if (url === LWS_TOKEN_ENDPOINT) return exchangeEndpoint(init); return provider.pod(input, init); }; @@ -324,11 +409,25 @@ const harness: SessionHarness = { refreshTokensIssued: true, rotates: true, idToken: true, + refreshIdToken: false, + idTokens: [], tokenRequests: [], registrations: [], issued: 0, pod: async () => new Response("pod"), }; + lws = { + configuration: { + issuer: ISSUER, + token_endpoint: LWS_TOKEN_ENDPOINT, + grant_types_supported: ["urn:ietf:params:oauth:grant-type:token-exchange"], + subject_token_types_supported: ["urn:ietf:params:oauth:token-type:id_token"], + }, + exchanges: [], + valid: new Set(), + refusesAll: false, + issued: 0, + }; vi.stubGlobal("fetch", fakeFetch); }, create() { @@ -1182,3 +1281,279 @@ describe("createSessionClient, over oauth4webapi", () => { await expect(client.fetch("https://pod.example/a")).rejects.toThrow("Failed to fetch"); }); }); + +describe("createSessionClient, on a Linked Web Storage storage", () => { + beforeEach(() => harness.reset()); + + async function logIn() { + await createSessionClient().login(LOGIN); + sendBack(); + const client = createSessionClient(); + await client.restore(); + return client; + } + + /** The requests the pod was sent, as their URL and Authorization; not the description's. */ + const sent = (pod: ReturnType) => + pod.mock.calls + .filter(([, init]) => new Headers(init?.headers).get("accept") !== "application/lws+cid") + .map(([input, init]) => [ + input instanceof Request ? input.url : String(input), + new Headers(init?.headers).get("authorization"), + ]); + + const RESOURCE = `${STORAGE}notes/a.ttl`; + + it("exchanges the ID token for the storage's token on its challenge, and sends the request again with it", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + const response = await client.fetch(RESOURCE, { + method: "PUT", + headers: { "content-type": "text/turtle" }, + body: "<> a <#t>.", + redirect: "error", + signal: new AbortController().signal, + }); + expect(response.status).toBe(200); + expect(sent(pod)).toEqual([ + [RESOURCE, "DPoP access-1"], + [RESOURCE, "Bearer lws-1"], + ]); + const retry = (pod.mock.calls.at(-1) as unknown as [string, RequestInit])[1]; + expect(retry).toMatchObject({ method: "PUT", body: "<> a <#t>.", redirect: "error" }); + expect(new Headers(retry.headers).get("content-type")).toBe("text/turtle"); + expect(new Headers(retry.headers).get("dpop")).toBeNull(); + const [exchange] = lws.exchanges; + expect(Object.fromEntries(exchange!.parameters)).toEqual({ + grant_type: "urn:ietf:params:oauth:grant-type:token-exchange", + resource: STORAGE, + subject_token: provider.idTokens[0], + subject_token_type: "urn:ietf:params:oauth:token-type:id_token", + client_id: "registered-1", + }); + expect(exchange?.dpop).toBeNull(); + }); + + it("sends the storage's token first afterwards, within the storage only", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + await client.fetch(RESOURCE); + pod.mockClear(); + await client.fetch(`${STORAGE}b`); + await client.fetch("https://pod.example/bob/x"); + await client.fetch("https://pod.example/alicex"); + expect(sent(pod)).toEqual([ + [`${STORAGE}b`, "Bearer lws-1"], + ["https://pod.example/bob/x", "DPoP access-1"], + ["https://pod.example/alicex", "DPoP access-1"], + ]); + expect(lws.exchanges).toHaveLength(1); + }); + + it("exchanges once for requests that meet the storage together", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + const answers = await Promise.all([client.fetch(RESOURCE), client.fetch(`${STORAGE}b`)]); + expect(answers.map(({ status }) => status)).toEqual([200, 200]); + expect(lws.exchanges).toHaveLength(1); + }); + + it("exchanges once more when the storage refuses a kept token, and passes the second refusal on", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + await client.fetch(RESOURCE); + lws.valid.clear(); + pod.mockClear(); + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 200 }); + expect(sent(pod)).toEqual([ + [RESOURCE, "Bearer lws-1"], + [RESOURCE, "Bearer lws-2"], + ]); + + lws.refusesAll = true; + pod.mockClear(); + const refused = await client.fetch(RESOURCE); + expect(refused.status).toBe(401); + expect(refused.headers.get("www-authenticate")).toContain('error="invalid_token"'); + expect(sent(pod)).toEqual([ + [RESOURCE, "Bearer lws-2"], + [RESOURCE, "Bearer lws-3"], + ]); + expect(lws.exchanges).toHaveLength(3); + }); + + it("passes the storage's refusal on when the authorization server refuses a new exchange", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + await client.fetch(RESOURCE); + lws.valid.clear(); + // The authorization server no longer takes the ID token. + provider.idTokens = []; + pod.mockClear(); + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); + expect(sent(pod)).toEqual([[RESOURCE, "Bearer lws-1"]]); + expect(lws.exchanges).toHaveLength(2); + }); + + it("asks the authorization server once for an ID token it refused, however many requests come", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + // The authorization server takes no ID token of this session's. + provider.idTokens = []; + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); + await expect(client.fetch(`${STORAGE}b`)).resolves.toMatchObject({ status: 401 }); + expect(lws.exchanges).toHaveLength(1); + expect(sent(pod).map(([, authorization]) => authorization)).toEqual([ + "DPoP access-1", + "DPoP access-1", + "DPoP access-1", + ]); + }); + + it("follows no redirect with the storage's token, which never leaves the storage", async () => { + const client = await logIn(); + const pod = lwsPod(); + provider.pod = pod; + await client.fetch(RESOURCE); + // The storage sends a request to an address outside it, on its own origin. + const redirecting = vi.fn(async (input, init) => + String(input) === `${STORAGE}moved` + ? new Response(null, { status: 302, headers: { location: "/bob/y" } }) + : pod(input, init), + ); + provider.pod = redirecting; + await expect(client.fetch(`${STORAGE}moved`)).resolves.toMatchObject({ status: 302 }); + await client.fetch(`${STORAGE}moved`, { redirect: "follow" }); + await client.fetch(`${STORAGE}moved`, { redirect: "error" }); + await client.fetch(new Request(`${STORAGE}moved`, { redirect: "error" })); + expect( + redirecting.mock.calls.map(([, init]) => [ + new Headers(init?.headers).get("authorization"), + init?.redirect, + ]), + ).toEqual([ + ["Bearer lws-1", "manual"], + ["Bearer lws-1", "manual"], + ["Bearer lws-1", "error"], + ["Bearer lws-1", "error"], + ]); + expect(redirecting.mock.calls.map(([input]) => String(input))).not.toContain( + "https://pod.example/bob/y", + ); + }); + + it("sends a body it can't send again once, and keeps what it learned for the next request", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + await expect(client.fetch(new Request(RESOURCE))).resolves.toMatchObject({ status: 401 }); + expect(lws.exchanges).toHaveLength(0); + await expect(client.fetch(new Request(RESOURCE))).resolves.toMatchObject({ status: 200 }); + lws.valid.clear(); + await expect(client.fetch(new Request(RESOURCE))).resolves.toMatchObject({ status: 401 }); + expect(sent(pod).map(([, authorization]) => authorization)).toEqual([ + "DPoP access-1", + "Bearer lws-1", + "Bearer lws-1", + ]); + }); + + it("sends the ID token to no authorization server but on its identity provider's origin", async () => { + const pod = lwsPod("https://as.example/"); + const client = await logIn(); + provider.pod = pod; + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); + expect(lws.exchanges).toEqual([]); + expect(pod.mock.calls.map(([input]) => String(input))).toEqual([RESOURCE]); + }); + + it("forgets the storages' tokens at logout", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + await client.fetch(RESOURCE); + await client.logout(); + const again = await logIn(); + pod.mockClear(); + await again.fetch(RESOURCE); + expect(sent(pod)).toEqual([ + [RESOURCE, "DPoP access-2"], + [RESOURCE, "Bearer lws-2"], + ]); + }); + + it("exchanges the ID token a restore's refresh brought", async () => { + provider.refreshIdToken = true; + storedSession(WEBID, "refresh-0"); + const client = createSessionClient(); + await client.restore(); + provider.pod = lwsPod(); + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 200 }); + expect(lws.exchanges[0]?.parameters.get("subject_token")).toBe(provider.idTokens[0]); + }); + + it("passes the challenge on when no refresh brings an ID token, and stops asking for one", async () => { + storedSession(WEBID, "refresh-0"); + const client = createSessionClient(); + await client.restore(); + const pod = lwsPod(); + provider.pod = pod; + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); + expect(provider.tokenRequests.map(({ parameters }) => parameters.get("grant_type"))).toEqual([ + "refresh_token", + "refresh_token", + ]); + expect(lws.exchanges).toEqual([]); + expect(client.webId()).toBe(WEBID); + }); + + it("refreshes an ID token about to expire, and keeps the old one when the refresh brings none", async () => { + provider.refreshIdToken = true; + const client = await logIn(); + provider.pod = lwsPod(); + vi.useFakeTimers({ toFake: ["Date"], now: Date.now() + 280_000 }); + try { + await client.fetch(RESOURCE); + expect(lws.exchanges[0]?.parameters.get("subject_token")).toBe(provider.idTokens[1]); + } finally { + vi.useRealTimers(); + } + + harness.reset(); + const other = await logIn(); + provider.pod = lwsPod(); + vi.useFakeTimers({ toFake: ["Date"], now: Date.now() + 280_000 }); + try { + await other.fetch(RESOURCE); + expect(provider.tokenRequests).toHaveLength(2); + expect(lws.exchanges[0]?.parameters.get("subject_token")).toBe(provider.idTokens[0]); + } finally { + vi.useRealTimers(); + } + }); + + it("goes as the access token's request once the storage's token and the ID token expired", async () => { + const pod = lwsPod(); + const client = await logIn(); + provider.pod = pod; + await client.fetch(RESOURCE); + pod.mockClear(); + vi.useFakeTimers({ toFake: ["Date"], now: Date.now() + 400_000 }); + try { + await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); + } finally { + vi.useRealTimers(); + } + // The refresh that brought no ID token renewed the access token; the request had the one it began with. + expect(sent(pod)).toEqual([[RESOURCE, "DPoP access-1"]]); + expect(lws.exchanges).toHaveLength(1); + }); +}); From e9ee464f27a8615645802babd851ea9f2fb181ed Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Thu, 5 Nov 2026 21:13:30 +0100 Subject: [PATCH 06/19] test(e2e): run the pod tests against CSS with LWS alone, and the bridge css-lws answers the app's Solid requests, so it shows nothing of what the app does where LWS alone is spoken. css-lws-only runs the same image with the fork's LWS-only configuration, as an advisory server. The tests' requests go through the bridge to an LWS server, as the app's do, and the contract's for a server SOLID_SERVER_URL names; its probe document is made only where none is, as the app makes one. The fork takes Solid's requests even with LWS alone, so a proxy that refuses them (npm run lws:strict) shows the bridge's own way of making them, for a contributor's runs, with E2E_SEEDS_AS_LWS=1 making a test's own seeds as LWS does. A compose file runs the fork's own LWS-only preset behind the journeys' TLS, for the journeys by hand. Refs: #594 --- .github/workflows/ci.yml | 2 +- .github/workflows/interop.yml | 2 +- e2e/journeys/css-lws/compose.yml | 44 +++++++ e2e/pod/contract.ts | 10 +- e2e/pod/globalSetup.ts | 4 +- e2e/pod/package.json | 3 +- e2e/pod/servers.test.ts | 3 + e2e/pod/servers.ts | 11 ++ e2e/pod/servers/css-lws-only/compose.yml | 21 ++++ e2e/pod/servers/css-lws/Dockerfile | 5 +- e2e/pod/servers/css-lws/css-lws-only-e2e.json | 38 ++++++ e2e/pod/setup.ts | 37 +++++- e2e/pod/strictLws.test.ts | 33 +++++ e2e/pod/strictLws.ts | 114 ++++++++++++++++++ 14 files changed, 316 insertions(+), 11 deletions(-) create mode 100644 e2e/journeys/css-lws/compose.yml create mode 100644 e2e/pod/servers/css-lws-only/compose.yml create mode 100644 e2e/pod/servers/css-lws/css-lws-only-e2e.json create mode 100644 e2e/pod/strictLws.test.ts create mode 100644 e2e/pod/strictLws.ts diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0e13a66a6..d20b0ceaf 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -578,7 +578,7 @@ jobs: # the run; on main, each runs to its end, saving its images. fail-fast: ${{ github.ref != 'refs/heads/main' }} matrix: - server: [css-8, pivot, nextcloud, jss, css-lws] + server: [css-8, pivot, nextcloud, jss, css-lws, css-lws-only] steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: diff --git a/.github/workflows/interop.yml b/.github/workflows/interop.yml index 4ac273d09..4a00ecff9 100644 --- a/.github/workflows/interop.yml +++ b/.github/workflows/interop.yml @@ -56,7 +56,7 @@ jobs: strategy: fail-fast: false matrix: - server: [css-8, pivot, nextcloud, jss, css-lws] + server: [css-8, pivot, nextcloud, jss, css-lws, css-lws-only] steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: diff --git a/e2e/journeys/css-lws/compose.yml b/e2e/journeys/css-lws/compose.yml new file mode 100644 index 000000000..ca5be475f --- /dev/null +++ b/e2e/journeys/css-lws/compose.yml @@ -0,0 +1,44 @@ +# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +# +# SPDX-License-Identifier: MIT + +# The Community Solid Server 8 with Linked Web Storage alone (the fork's +# own config/lws.json: files, a storage for each pod), from the image +# e2e/pod/servers/css-lws/ builds (`npm run servers -w @solid-memo/e2e-pod +# -- css-lws`), behind css/'s TLS, for a contributor to run the journeys +# on by hand (docs/lws.md): `E2E_PORT= docker compose -f +# e2e/journeys/css-lws/compose.yml up -d`, then +# `JOURNEY_SERVER_URL=https://127.0.0.1:/ npm run journeys` with the +# app built with `SOLID_CLIENT=solid-memo-client`. CI runs none. +services: + tls: + image: caddy:2.11.7@sha256:36c24ad91a7ff7bc4c87fe22bfa8df3c4b18da0773d73fc12ee2c52b98c0bb7e + command: ["caddy", "run", "--config", "/etc/caddy/Caddyfile", "--adapter", "caddyfile"] + environment: { E2E_PORT: "${E2E_PORT:?}" } + volumes: ["../css/Caddyfile:/etc/caddy/Caddyfile:ro"] + tmpfs: ["/data:uid=1000,gid=1000", "/config:uid=1000,gid=1000"] + ports: ["127.0.0.1:${E2E_PORT:?}:${E2E_PORT:?}"] + user: "1000:1000" + cap_drop: [ALL] + # Its binary carries this file capability, without which it cannot run. + cap_add: [NET_BIND_SERVICE] + security_opt: ["no-new-privileges:true"] + mem_limit: 256m + pids_limit: 256 + server: + image: localhost/solid-memo-e2e-css-lws:local + pull_policy: never + command: ["-c", "config/lws.json", "-f", "/data", "-b", "https://127.0.0.1:${E2E_PORT:?}/", "-l", "info"] + network_mode: "service:tls" + depends_on: [tls] + # Caddy's CA is made as it starts, and this server is thrown away after + # the run: it trusts whatever certificate its own requests meet, the + # LWS token endpoint's discovery of its own OpenID provider among them. + environment: { NODE_TLS_REJECT_UNAUTHORIZED: "0" } + tmpfs: ["/data:uid=1000,gid=1000,mode=0700"] + user: node + init: true + cap_drop: [ALL] + security_opt: ["no-new-privileges:true"] + mem_limit: 1g + pids_limit: 512 diff --git a/e2e/pod/contract.ts b/e2e/pod/contract.ts index f14be35c8..4e3f137e1 100644 --- a/e2e/pod/contract.ts +++ b/e2e/pod/contract.ts @@ -13,12 +13,18 @@ const LDP_CONTAINS = "http://www.w3.org/ns/ldp#contains"; * lists it at the URL it was written to (so the server knows the URL it * is reached at: a wrong base URL, port or trailing slash shows here, * not as every test failing), and it deletes. Throws saying which failed. + * The requests go through `fetch`: the bridge to a Linked Web Storage + * server (lwsBridge.ts) for a server a contributor names, which may be one. */ -export async function checkContract(root: string): Promise { +export async function checkContract( + root: string, + fetch: typeof globalThis.fetch = globalThis.fetch, +): Promise { const url = new URL(`probe-${randomUUID()}.ttl`, root).href; const put = await fetch(url, { method: "PUT", - headers: { "content-type": "text/turtle" }, + // Made as the app makes a document: only where none is. + headers: { "content-type": "text/turtle", "if-none-match": "*" }, body: `<#a> <#b> "1" .`, }).catch((error: Error) => { throw new Error( diff --git a/e2e/pod/globalSetup.ts b/e2e/pod/globalSetup.ts index 15dab15e4..fc7a8c314 100644 --- a/e2e/pod/globalSetup.ts +++ b/e2e/pod/globalSetup.ts @@ -3,6 +3,7 @@ // SPDX-License-Identifier: MIT import type { TestProject } from "vitest/node"; +import { lwsBridge } from "@solid-memo/solid/lwsBridge"; import { checkContract } from "./contract"; import { serversNamed, @@ -36,7 +37,8 @@ export default async function setup(project: TestProject) { const given = process.env.SOLID_SERVER_URL; if (given !== undefined && given !== "") { const url = given.endsWith("/") ? given : `${given}/`; - await checkContract(url); + // Made as the app makes its requests, through the bridge to a Linked Web Storage server it may be (setup.ts). + await checkContract(url, lwsBridge(fetch)); project.provide("solidServers", [ { id: "external", tier: "advisory", name: process.env.SOLID_SERVER_NAME || given, url }, ]); diff --git a/e2e/pod/package.json b/e2e/pod/package.json index 4b9b441c1..0d8c3465d 100644 --- a/e2e/pod/package.json +++ b/e2e/pod/package.json @@ -13,7 +13,8 @@ "test:pod": "vitest run", "pod": "node pod.ts", "servers": "node servers.ts prepare", - "pod:clean": "node servers.ts clean" + "pod:clean": "node servers.ts clean", + "lws:strict": "node strictLws.ts" }, "dependencies": { "@solid-memo/application": "*", diff --git a/e2e/pod/servers.test.ts b/e2e/pod/servers.test.ts index ffef4d1d8..d2df82825 100644 --- a/e2e/pod/servers.test.ts +++ b/e2e/pod/servers.test.ts @@ -234,6 +234,9 @@ describe("the names in the tests", () => { "css-lws": expect.stringMatching( /^Community Solid Server with LWS 8\.\d+\.\d+(-[\w.]+)? \([0-9a-f]{8}\)$/, ), + "css-lws-only": expect.stringMatching( + /^Community Solid Server with LWS only 8\.\d+\.\d+(-[\w.]+)? \([0-9a-f]{8}\)$/, + ), }); }); }); diff --git a/e2e/pod/servers.ts b/e2e/pod/servers.ts index 6fd95daa0..70644d92d 100644 --- a/e2e/pod/servers.ts +++ b/e2e/pod/servers.ts @@ -188,6 +188,17 @@ export const SERVERS = { version: () => `${dockerfileArg("css-lws", "CSS_LWS_VERSION")} (${dockerfileArg("css-lws", "CSS_LWS_SHA").slice(0, 8)})`, }, + "css-lws-only": { + label: "Community Solid Server with LWS only", + tier: "advisory", + internalPort: 3000, + podPath: "", + readyPath: "", + startTimeoutMs: 60_000, + // css-lws's image, run with LWS alone: what the app does where Solid's own requests are not taken (docs/lws.md). + version: () => + `${dockerfileArg("css-lws", "CSS_LWS_VERSION")} (${dockerfileArg("css-lws", "CSS_LWS_SHA").slice(0, 8)})`, + }, } satisfies Record; export type ServerId = keyof typeof SERVERS; diff --git a/e2e/pod/servers/css-lws-only/compose.yml b/e2e/pod/servers/css-lws-only/compose.yml new file mode 100644 index 000000000..cb3fd6790 --- /dev/null +++ b/e2e/pod/servers/css-lws-only/compose.yml @@ -0,0 +1,21 @@ +# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +# +# SPDX-License-Identifier: MIT + +# The Community Solid Server 8 with Linked Web Storage alone, built from +# ../css-lws/Dockerfile, in memory, its root open to anyone +# (css-lws-only-e2e.json): an advisory server (docs/testing.md). The +# harness (../../servers.ts) sets E2E_PORT, the port on 127.0.0.1 it is +# reached at. +services: + server: + build: ../css-lws + image: localhost/solid-memo-e2e-css-lws-only:local + pull_policy: never + command: ["-c", "config/css-lws-only-e2e.json", "-b", "http://127.0.0.1:${E2E_PORT:?}/", "-l", "warn"] + ports: ["127.0.0.1:${E2E_PORT:?}:3000"] + init: true + cap_drop: [ALL] + security_opt: ["no-new-privileges:true"] + mem_limit: 1g + pids_limit: 512 diff --git a/e2e/pod/servers/css-lws/Dockerfile b/e2e/pod/servers/css-lws/Dockerfile index 0c9ae9361..721e0a1e3 100644 --- a/e2e/pod/servers/css-lws/Dockerfile +++ b/e2e/pod/servers/css-lws/Dockerfile @@ -6,7 +6,8 @@ # end-to-end tests (docs/testing.md): the branch of jeswr's fork that adds # LWS to CSS 8.0.0-alpha.3 (pull request 126 there), at a commit # (CSS_LWS_SHA), as no release or image of it is published. It is built -# without its dependencies' install scripts, then run with css-lws-e2e.json. +# without its dependencies' install scripts, then run with css-lws-e2e.json +# (css-lws) or css-lws-only-e2e.json (css-lws-only). FROM node:22.23.4-bookworm-slim@sha256:4b150b48b4116e2a22f2c39cc27025fbf4f153e2078fd3b14d159fdb5c7279da AS build # Renovate proposes newer commits of the branch for a person to approve (renovate.json5). # renovate: datasource=git-refs depName=https://github.com/jeswr/CommunitySolidServer branch=feat/lws-next-major @@ -30,6 +31,6 @@ COPY --from=build /community-server/config ./config COPY --from=build /community-server/dist ./dist COPY --from=build /community-server/node_modules ./node_modules COPY --from=build /community-server/templates ./templates -COPY css-lws-e2e.json ./config/css-lws-e2e.json +COPY css-lws-e2e.json css-lws-only-e2e.json ./config/ USER node ENTRYPOINT ["node", "bin/server.js"] diff --git a/e2e/pod/servers/css-lws/css-lws-only-e2e.json b/e2e/pod/servers/css-lws/css-lws-only-e2e.json new file mode 100644 index 000000000..f8c1517a5 --- /dev/null +++ b/e2e/pod/servers/css-lws/css-lws-only-e2e.json @@ -0,0 +1,38 @@ +{ + "comment": "The configuration of the fork's config/lws.json (Linked Web Storage alone, with WAC), in memory with the root as the storage, as the Community Solid Server's config/default.json has them. The root is initialized empty and open to anyone, as the end-to-end tests need (docs/testing.md).", + "@context": "https://linkedsoftwaredependencies.org/bundles/npm/@solid/community-server/^8.0.0/components/context.jsonld", + "import": [ + "css:config/app/init/initialize-root.json", + "css:config/app/main/default.json", + "css:config/app/variables/default.json", + "css:config/http/handler/lws.json", + "css:config/http/middleware/default.json", + "css:config/http/notifications/all.json", + "css:config/http/server-factory/http.json", + "css:config/http/static/default.json", + "css:config/identity/access/public.json", + "css:config/identity/email/default.json", + "css:config/identity/handler/default.json", + "css:config/identity/lws/default.json", + "css:config/identity/oidc/default.json", + "css:config/identity/ownership/token.json", + "css:config/identity/pod/static.json", + "css:config/ldp/authentication/lws.json", + "css:config/ldp/authorization/webacl.json", + "css:config/ldp/handler/lws.json", + "css:config/ldp/metadata-parser/lws.json", + "css:config/ldp/metadata-writer/lws.json", + "css:config/ldp/modes/lws.json", + "css:config/storage/backend/memory.json", + "css:config/storage/key-value/resource-store.json", + "css:config/storage/location/root.json", + "css:config/storage/middleware/lws.json", + "css:config/util/auxiliary/acl.json", + "css:config/util/identifiers/suffix.json", + "css:config/util/index/default.json", + "css:config/util/logging/winston.json", + "css:config/util/representation-conversion/lws.json", + "css:config/util/resource-locker/memory.json", + "css:config/util/variables/default.json" + ] +} diff --git a/e2e/pod/setup.ts b/e2e/pod/setup.ts index d4acd1266..06e8afbc8 100644 --- a/e2e/pod/setup.ts +++ b/e2e/pod/setup.ts @@ -2,6 +2,7 @@ // // SPDX-License-Identifier: MIT +import { lwsBridge } from "@solid-memo/solid/lwsBridge"; import { closingConnections } from "./connections.ts"; const send = closingConnections(globalThis.fetch); @@ -12,7 +13,37 @@ const send = closingConnections(globalThis.fetch); * module keeps the fetch it was given. None follows a redirect, as none * the app sends with the user's credentials does (authFetch.ts), so a * server that answers a pod request with one fails here. A scripted login - * reads each redirect it gets, by asking for "manual" (signedIn.ts). + * reads each redirect it gets, by asking for "manual" (signedIn.ts). A request a + * Linked Web Storage server does not take as the app makes it is made as + * LWS has it, as the app's are (lwsBridge.ts, docs/lws.md); to any other + * server it is sent as it is. */ -globalThis.fetch = (input, init) => - send(input, { ...init, redirect: init?.redirect === "manual" ? "manual" : "error" }); +const bridged = lwsBridge((input, init) => + send(input, { ...init, redirect: init?.redirect === "manual" ? "manual" : "error" }), +); + +/** + * With E2E_SEEDS_AS_LWS=1, for a run on a server that takes only LWS + * (strictLws.ts): what a test writes with a plain PUT, as another app + * writes its data, and the server answers 404, not there yet, is made + * as LWS makes a resource, as the app makes a document (with + * `If-None-Match: *`, which every creation of the app's sends). The + * app's own requests are never so made again: none is a plain PUT of + * something not there (docs/lws.md). + */ +const seedsAsLws = process.env.E2E_SEEDS_AS_LWS === "1"; + +globalThis.fetch = async (input, init) => { + const response = await bridged(input, init); + const headers = new Headers(init?.headers); + if ( + !seedsAsLws || + init?.method !== "PUT" || + headers.has("If-None-Match") || + headers.has("If-Match") || + response.status !== 404 + ) + return response; + headers.set("If-None-Match", "*"); + return bridged(input, { ...init, headers }); +}; diff --git a/e2e/pod/strictLws.test.ts b/e2e/pod/strictLws.test.ts new file mode 100644 index 000000000..b3a9ebb93 --- /dev/null +++ b/e2e/pod/strictLws.test.ts @@ -0,0 +1,33 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { describe, expect, it } from "vitest"; +import { ruleFor } from "./strictLws.ts"; + +describe("the strict LWS proxy's rules", () => { + it("lists a container only as LWS does, and refuses Turtle alone", () => { + expect(ruleFor("GET", "/a/", "text/turtle")).toEqual({ refuse: 406 }); + expect(ruleFor("HEAD", "/a/", "text/turtle, */*;q=0.1")).toEqual({ + send: true, + accept: "application/lws+json", + }); + expect(ruleFor("GET", "/a/", "application/lws+json")).toEqual({ + send: true, + accept: "application/lws+json", + }); + expect(ruleFor("GET", "/a/", undefined)).toEqual({ + send: true, + accept: "application/lws+json", + }); + expect(ruleFor("GET", "/", "application/lws+cid")).toEqual({ send: true }); + }); + + it("refuses every PATCH and a PUT of a resource not there, and sends the rest on", () => { + expect(ruleFor("PATCH", "/a/b.ttl", undefined)).toEqual({ refuse: 415 }); + expect(ruleFor("PUT", "/a/b.ttl", undefined)).toEqual({ sendIfThere: true }); + expect(ruleFor("GET", "/a/b.ttl", "text/turtle")).toEqual({ send: true }); + expect(ruleFor("POST", "/a/", undefined)).toEqual({ send: true }); + expect(ruleFor("DELETE", "/a/b.ttl", undefined)).toEqual({ send: true }); + }); +}); diff --git a/e2e/pod/strictLws.ts b/e2e/pod/strictLws.ts new file mode 100644 index 000000000..2abbb2448 --- /dev/null +++ b/e2e/pod/strictLws.ts @@ -0,0 +1,114 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +/** + * `npm run lws:strict -w @solid-memo/e2e-pod -- `: a proxy + * on 127.0.0.1: in front of a Linked Web Storage server that also + * takes the Solid Protocol's requests (the Community Solid Server with + * LWS, css-lws-only), which refuses what LWS Protocol 1.0 does not ask a + * server to take, so that the end-to-end tests show what the app does + * on a server that takes only LWS (docs/lws.md): a container listed in + * Turtle (406; an LWS server lists one as `application/lws+json`), a + * PUT of a resource not there (404; LWS makes one with a POST to its + * container), and any PATCH (415; LWS requires only JSON Patch, which + * the app does not send). The server must be told the proxy's URL as + * its base URL. For a contributor's own runs; CI runs none. + */ +import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; + +/** What the proxy does with a request: send it on (with this Accept, if given), ask first whether it is there, or refuse it. */ +export type Rule = + | { send: true; accept?: string } + | { sendIfThere: true } + | { refuse: 404 | 406 | 415 }; + +const LWS_JSON = "application/lws+json"; + +/** The rule for a request: its method, its path, and its Accept header. */ +export function ruleFor(method: string, path: string, accept: string | undefined): Rule { + if ((method === "GET" || method === "HEAD") && path.endsWith("/")) { + const asked = accept ?? "*/*"; + // A storage's description, at its URI, which is its root container's too, is sent on as asked for. + if (asked.includes("application/lws+cid")) return { send: true }; + if (asked.includes(LWS_JSON) || asked.includes("*/*")) return { send: true, accept: LWS_JSON }; + return { refuse: 406 }; + } + if (method === "PATCH") return { refuse: 415 }; + if (method === "PUT") return { sendIfThere: true }; + return { send: true }; +} + +/** The headers of a response a proxy may pass on as they are (fetch has undone its encoding and length). */ +const DROPPED = new Set([ + "content-encoding", + "content-length", + "transfer-encoding", + "connection", + "keep-alive", +]); + +async function bodyOf(request: IncomingMessage): Promise { + const chunks: Uint8Array[] = []; + for await (const chunk of request) chunks.push(new Uint8Array(chunk as Buffer)); + return chunks.length === 0 ? undefined : new Blob(chunks); +} + +function headersOf(request: IncomingMessage, accept?: string): Headers { + const headers = new Headers(); + for (const [name, value] of Object.entries(request.headers)) + if (value !== undefined && name !== "host" && name !== "connection") + headers.set(name, Array.isArray(value) ? value.join(", ") : value); + // The server names its resources by the host it is asked at, the proxy's (its base URL). + headers.set("x-forwarded-host", request.headers.host ?? ""); + headers.set("x-forwarded-proto", "http"); + if (accept !== undefined) headers.set("accept", accept); + return headers; +} + +async function proxy(server: string, request: IncomingMessage, response: ServerResponse) { + const method = request.method ?? "GET"; + const target = new URL(request.url ?? "/", server).href; + const rule = ruleFor(method, new URL(target).pathname, request.headers.accept); + const body = await bodyOf(request); + if ("refuse" in rule) { + response.writeHead(rule.refuse, { "access-control-allow-origin": "*" }).end(); + return; + } + if ("sendIfThere" in rule) { + const there = await fetch(target, { method: "HEAD", headers: headersOf(request) }); + if (there.status === 404) { + response.writeHead(404).end(); + return; + } + } + const answer = await fetch(target, { + method, + headers: headersOf(request, "accept" in rule ? rule.accept : undefined), + body, + redirect: "manual", + }); + const headers: Record = {}; + answer.headers.forEach((value, name) => { + if (!DROPPED.has(name)) headers[name] = [...(headers[name] ?? []), value]; + }); + response.writeHead(answer.status, headers); + response.end(method === "HEAD" ? undefined : Buffer.from(await answer.arrayBuffer())); +} + +if (import.meta.main) { + const [server, port] = process.argv.slice(2); + if (server === undefined || port === undefined) { + console.error("Usage: node strictLws.ts "); + process.exit(2); + } + createServer((request, response) => { + proxy(server, request, response).catch((error: Error) => { + response.writeHead(502).end(error.message); + }); + }).listen(Number(port), "127.0.0.1", () => + console.log( + `A server that takes only LWS, in front of ${server}, at http://127.0.0.1:${port}/`, + ), + ); +} From 60cab937328f200e4393b4dabc795ef0a62eaaf8 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Thu, 5 Nov 2026 21:13:30 +0100 Subject: [PATCH 07/19] docs: explain how Solid Memo works on a Linked Web Storage server A contributor meeting the bridge, the storage description or the token exchange needs to know why each is made the way it is, and what LWS can't give the app yet, such as a release anyone can read. The testing page names css-lws-only and how the bridge's own way of making requests is tried, and the standards page names LWS and the OAuth specifications it uses. Refs: #594 --- docs/docs.ttl | 7 +++ docs/lws.md | 145 ++++++++++++++++++++++++++++++++++++++++++++++ docs/standards.md | 2 + docs/testing.md | 23 ++++++-- 4 files changed, 173 insertions(+), 4 deletions(-) create mode 100644 docs/lws.md diff --git a/docs/docs.ttl b/docs/docs.ttl index e66aba8e9..82f1d43d7 100644 --- a/docs/docs.ttl +++ b/docs/docs.ttl @@ -77,6 +77,13 @@ schema:learningResourceType "reference" ; dcterms:abstract "the second Solid client, on no Inrupt library: what it replaces, switching to it with SOLID_CLIENT=solid-memo-client, how it signs in and keeps the session, the public-read grant, how it differs from Inrupt's, and its tests."@en . + + a schema:TechArticle ; + schema:isPartOf <#the-code> ; + schema:position 46 ; + schema:learningResourceType "explanation" ; + dcterms:abstract "how the app works on a Linked Web Storage server: finding the storage, the bridge that makes the app's Solid requests as LWS has them, signing in with a token exchange, and what LWS can't give the app yet."@en . + a schema:TechArticle ; schema:isPartOf <#the-code> ; diff --git a/docs/lws.md b/docs/lws.md new file mode 100644 index 000000000..704eaaacd --- /dev/null +++ b/docs/lws.md @@ -0,0 +1,145 @@ +# Linked Web Storage + +Solid Memo is written for the Solid Protocol. The W3C's Linked Web +Storage (LWS) Working Group is writing its successor, +[LWS Protocol 1.0](https://www.w3.org/TR/lws10-core/), still a Working +Draft. This page explains how the app works on a storage that speaks +LWS, why each part is made the way it is, and what LWS can't give it +yet. The servers it's tested on are in [testing.md](testing.md). + +## What differs + +An app that works on a Solid pod asks a few things of it that LWS +doesn't ask a server to give: + +| The app needs | Solid Protocol | LWS | +|---|---|---| +| The user's storage | `pim:storage` in the profile, or the folder typed `pim:Storage` | A `Link` with `rel="https://www.w3.org/ns/lws#storage"` on every answer, and the storage's description (`application/lws+cid`), which names its root container | +| A folder's contents | Turtle, `ldp:contains` | `application/lws+json`, page by page | +| A new document at a name the app chooses | PUT with `If-None-Match: *` | POST to its container, which names it; the name asked for is a hint | +| An edit of a document | SPARQL Update or N3 Patch | JSON Patch, or the whole document again with PUT | +| Sign-in | Solid-OIDC, a DPoP-bound access token | The OpenID Connect ID token exchanged for a Bearer token for each storage | +| A release anyone can read | Web Access Control or Access Control Policies | Nothing yet | + +A server can speak both. The Community Solid Server with LWS +(`css-lws`) does, and answers every request the app makes as a Solid +pod does. One that speaks LWS alone gets the app's requests made as LWS +has them, as below. + +## Finding the storage + +The app reads `pim:storage` from the user's profile first. Without one, +it walks from the WebID's document up its folders and asks each with a +HEAD. A Solid folder typed `pim:Storage` is the storage. An LWS answer +links its storage instead; the app reads that storage's description +and takes the root container it names, if the WebID's document is in +it ([lwsStorage.ts](../packages/solid/src/lwsStorage.ts), +`solidStorageGateway.ts`). + +A deck's folder that's a storage's root holds the user's other data +too, so the app claims none of its documents +([deckPlace.ts](../packages/solid/src/deckPlace.ts)). On LWS, a folder +is a root when the storage description names it as one: the storage +link alone doesn't say so, since every resource in the storage carries +it. + +**Why the app asks for Turtle first.** On the Community Solid Server +with LWS, the storage's URI is its root container's too. Asked for +anything, it answers with the storage's description, which has none of +the Solid links: no `pim:Storage` type, no link to the ACL. Each HEAD +that reads a resource's links asks for `text/turtle, */*;q=0.1` +(`TURTLE_FIRST`, `client/linkHeader.ts`). A server that has Turtle +answers with it, its Solid links kept. A server that lists a folder +only as LWS has it answers too, where Turtle alone would get +a 406. + +## The bridge + +[lwsBridge.ts](../packages/solid/src/lwsBridge.ts) sits below the +fences (`podFence.ts`, `ownPodGate.ts`), on the user's fetch and on the +fetch as no one ([appUseCases.ts](../packages/composition/src/appUseCases.ts)). +It sends each request as the app makes it, and makes one as LWS has it +only when the server's answer asks for that: + +- **A folder's listing.** An answer in `application/lws+json` is read + as LWS gives it. A 406 to Turtle, or Turtle in the LWS profile, which + lists no `ldp:contains`, from a server that links its storage, gets + the listing asked for again in `application/lws+json`. The bridge + follows each page's `next` link on the same origin, and answers with + the Turtle listing the app reads, the first page's ETag and links + kept. +- **A new document or folder.** A PUT with `If-None-Match: *` answered + 404, 405, 415 or 501 by a server that links its storage is made + another way. The bridge makes each missing folder above it, then + POSTs the document to its folder, the name asked for in `Slug` as + the address spells it. A + resource the server names otherwise is deleted again, and the + creation is refused with 501. When the server made it where it + wouldn't say, or outside the folder, or the delete fails, the + creation is refused with 502, and nothing is deleted or tried again. +- **An edit.** A SPARQL Update PATCH (`INSERT DATA`, `DELETE DATA`) + answered the same way: the bridge reads the document, applies the + change, and PUTs the whole document with `If-Match`. A triple to + delete that isn't there fails the edit with 409, as a Solid server's + PATCH does. A PATCH that had no `If-Match`, the answer log's insert, + is written against the ETag read, never a weak one, and tried again + up to five times when another write came first, so every concurrent + answer is kept. A document not there yet is made as above. + +**Why it reacts to the server's answer.** A Solid server takes each +request as it's made, so it gets one request, as before. Which server +speaks which protocol is never configured: the server's answer says it. +Whether a server is an LWS server is asked by a HEAD of the resource or +the nearest folder above it that's there, whose answer must link its +storage. + +**Why the fences stay above it.** The pod fence and the credential gate +check each request as the app makes it, a write's target among them. +What the bridge sends besides goes to the same resource, the folders +above it, a resource its POST made in the folder, or the pages the +server links on its own origin. + +**Why a name the server changes is refused.** Every address in an +instance is the app's to choose ([data-model.md](data-model.md)): a +catalog names its decks' documents by address, and a type index names +the instance. A storage that names its resources itself can't hold an +instance until the app records where each resource went. LWS has an +open issue for a creation at a name the client chooses +([w3c/lws-protocol#173](https://github.com/w3c/lws-protocol/issues/173)). + +## Sign-in + +An LWS storage takes a Bearer token that its authorization server +issues for the OpenID Connect ID token. The app's own Solid client +([solid-memo-client.md](solid-memo-client.md#linked-web-storage)) does +that exchange when a storage answers 401 with an LWS challenge, and +keeps one token for each storage. Inrupt's sign-in client gives the +app the ID token only in the address that ends the session, so a build that signs in to an LWS storage is made with +`SOLID_CLIENT=solid-memo-client`. + +## Trying it by hand + +The end-to-end tests send no credentials, so sign-in on an LWS storage +is tried by hand, with the user journeys +([e2e/journeys/css-lws/compose.yml](../e2e/journeys/css-lws/compose.yml)). +The fork's image, which `npm run servers -w @solid-memo/e2e-pod -- +css-lws` builds, runs its own `config/lws.json` behind the journeys' +TLS, each account's pod a storage of its own. The app is built with +`SOLID_CLIENT=solid-memo-client`, and `JOURNEY_SERVER_URL` names the +server. A journey that writes to the pod as another app +(`harness/otherApp.ts`) fails there: its Solid-OIDC token is no LWS +storage's. + +## What LWS can't give the app yet + +- **A release anyone can read.** LWS defines no access control. The app + writes a release's ACL or ACR where the server has one; the Community + Solid Server with LWS keeps Web Access Control, so a release is made + public there. On a server with LWS alone, a release is published + readable by its owner alone, and asking to make it public fails + with `publicAccessRefused` + ([releasePublishing.ts](../packages/application/src/releasePublishing.ts)). +- **Live updates.** LWS has only webhooks, which a page in a browser + can't receive. The app doesn't use notifications. +- **A type index the server keeps.** The app keeps its own type index + documents, which an LWS server holds as any other data. diff --git a/docs/standards.md b/docs/standards.md index 74e544017..22b89bdae 100644 --- a/docs/standards.md +++ b/docs/standards.md @@ -20,6 +20,8 @@ Every specification below is open; the drafts and de facto ones are in | [Type Indexes](https://solid.github.io/type-indexes/) | 1.0.0 | Editor's draft | Finding and registering an instance: [`typeIndex.ts`](../packages/solid/src/typeIndex.ts) ([data-model.md](data-model.md#registrations)) | Yes (draft) | | [Web Access Control](https://solidproject.org/TR/wac) and [Access Control Policy](https://solidproject.org/TR/acp) | WAC 1.0.0; ACP 0.9.0 | W3C Solid Community Group drafts | Making a published release public: [`solidReleasePublisher.ts`](../packages/solid/src/solidReleasePublisher.ts), through `@inrupt/solid-client`'s universal access in [`client/access.ts`](../packages/solid/src/client/access.ts), which speaks whichever the pod uses ([boundaries.md](boundaries.md)) | Yes (draft) | | [Linked Data Platform](https://www.w3.org/TR/ldp/) | 1.0 | W3C Recommendation | Containers in the pod and the guest's pod: [`localPod.ts`](../packages/solid/src/localPod.ts) ([guest-mode.md](guest-mode.md)) | Yes | +| [Linked Web Storage Protocol](https://www.w3.org/TR/lws10-core/) and its [OpenID Connect authentication suite](https://www.w3.org/TR/lws10-authn-openid/) | 1.0 | W3C Working Draft | A storage that speaks LWS: finding it ([`lwsStorage.ts`](../packages/solid/src/lwsStorage.ts)), the bridge that makes the app's requests as LWS has them ([`lwsBridge.ts`](../packages/solid/src/lwsBridge.ts)), and its sign-in ([`lwsAuth.ts`](../packages/solid-memo-client/src/lwsAuth.ts)) ([lws.md](lws.md)) | Yes (draft) | +| [OAuth 2.0 Token Exchange](https://www.rfc-editor.org/rfc/rfc8693) and [Protected Resource Metadata](https://www.rfc-editor.org/rfc/rfc9728) | RFC 8693 (2020); RFC 9728 (2025) | IETF Proposed Standards | The ID token exchanged for an LWS storage's token, and the challenge that leads to it (`lwsAuth.ts`, [solid-memo-client.md](solid-memo-client.md#linked-web-storage)) | Yes | ## Linked data diff --git a/docs/testing.md b/docs/testing.md index a289524c7..29ebcb97c 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -75,6 +75,7 @@ or on Ctrl-C. The test names say which server and version: | `nextcloud` | Solid-Nextcloud: Nextcloud 32 with the Solid app, on SQLite, the pod `apps/solid/~alice/storage/` | built here from a commit of the app ([servers/nextcloud/](../e2e/pod/servers/nextcloud/Dockerfile)), installed on every start, alice's pod opened by a hook | advisory | | `jss` | JavaScript Solid Server, files in the container, open to anyone (`--public`, which skips its access control) | built here from its lockfile ([servers/jss/](../e2e/pod/servers/jss/Dockerfile)), unmodified and pushed nowhere (it's AGPL) | advisory | | `css-lws` | Community Solid Server 8 with Linked Web Storage (LWS): the Solid Protocol and LWS side by side, in memory | built here from a commit of a fork not yet merged ([servers/css-lws/](../e2e/pod/servers/css-lws/Dockerfile)), with a configuration of its own, its root open | advisory | +| `css-lws-only` | The same with LWS alone, in memory | css-lws's image ([servers/css-lws-only/](../e2e/pod/servers/css-lws-only/compose.yml)), with a configuration of its own, its root open | advisory | A **blocking** server's tests gate CI, and so deploys and Renovate's merges: every test passes, or skips for a reason it gives. An @@ -108,10 +109,11 @@ Community Solid Server before any release has it: the branch [jeswr/CommunitySolidServer](https://github.com/jeswr/CommunitySolidServer/pull/126), on the same 8.0.0-alpha.3 as css-8, so a failure only css-lws shows comes from the LWS changes. Its configuration is the fork's -`config/solid-lws.json` in memory, with the root as the storage. The -fork's LWS-only configuration isn't tested: it treats the app's -Solid-OIDC requests as anonymous, so the app can't sign in there. The -image is built from code nobody has reviewed, on CI's runners and on the +`config/solid-lws.json` in memory, with the root as the storage. +css-lws-only runs the same image with the fork's `config/lws.json` +instead: LWS alone. The tests send no credentials, so neither shows +how the app signs in to an LWS storage; [lws.md](lws.md) says how +that's tried by hand. The image is built from code nobody has reviewed, on CI's runners and on the machine of anyone who runs `SOLID_SERVERS=all`. Its dependencies' install scripts don't run, and its final stage keeps only what the server runs. GitHub's dependency review doesn't see the lockfile it @@ -121,6 +123,19 @@ Community Solid Server release has LWS, or when its commit can't be fetched any more: a commit that can't be fetched fails the `contract` job of every pull request. +Every request of the tests goes through the bridge to an LWS server +(`lwsBridge.ts`, `setup.ts`), as the app's do, +and the contract's too for a server `SOLID_SERVER_URL` names. A Solid +server takes each request as it's made, so the bridge sends it on as +it is. The fork takes the Solid Protocol's requests even with LWS +alone, so the bridge's own way of making them shows only on a server +that refuses them. `npm run lws:strict -w @solid-memo/e2e-pod -- + ` runs one ([strictLws.ts](../e2e/pod/strictLws.ts)): a +proxy that refuses a folder's listing in Turtle, a PUT of a resource +not there, and every PATCH. With `E2E_SEEDS_AS_LWS=1`, a test's own +seed, a plain PUT of a document not there, is made as LWS makes one. CI +runs neither. + Each server is a compose file, `e2e/pod/servers//compose.yml`, asking only for `E2E_PORT` (the port on 127.0.0.1 the harness picked, which the server must be told) and `E2E_SECRET` (a password made up for the run). It publishes on From 0c9ad1ca0ed554de00327609536259bf794608b8 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Fri, 6 Nov 2026 13:22:24 +0100 Subject: [PATCH 08/19] fix(e2e): keep the strict LWS proxy on the server it fronts A request line may name a host of its own, as an absolute target or as //host/, and new URL(request.url, server) then went to that host, so anything that could reach the port could send requests elsewhere through the proxy, and read their answers. CodeQL flags it as server-side request forgery. Only the request's path and query are taken now, onto the server's own URL. Refs: #594 --- e2e/pod/strictLws.test.ts | 22 +++++++++++++++++++++- e2e/pod/strictLws.ts | 18 ++++++++++++++++-- 2 files changed, 37 insertions(+), 3 deletions(-) diff --git a/e2e/pod/strictLws.test.ts b/e2e/pod/strictLws.test.ts index b3a9ebb93..220d9be15 100644 --- a/e2e/pod/strictLws.test.ts +++ b/e2e/pod/strictLws.test.ts @@ -3,7 +3,7 @@ // SPDX-License-Identifier: MIT import { describe, expect, it } from "vitest"; -import { ruleFor } from "./strictLws.ts"; +import { ruleFor, targetOf } from "./strictLws.ts"; describe("the strict LWS proxy's rules", () => { it("lists a container only as LWS does, and refuses Turtle alone", () => { @@ -31,3 +31,23 @@ describe("the strict LWS proxy's rules", () => { expect(ruleFor("DELETE", "/a/b.ttl", undefined)).toEqual({ send: true }); }); }); + +describe("the strict LWS proxy's target", () => { + const server = "http://127.0.0.1:3000/"; + + it("keeps a request's path and query", () => { + expect(targetOf(server, "/a/b.ttl?c=d").href).toBe("http://127.0.0.1:3000/a/b.ttl?c=d"); + expect(targetOf(server, "/a/../b/").href).toBe("http://127.0.0.1:3000/b/"); + }); + + it("stays on the server, whatever host a request line names", () => { + for (const requestTarget of [ + "http://elsewhere.example/a", + "//elsewhere.example/a", + "/\\elsewhere.example/a", + "/.//elsewhere.example/a", + "http://user:password@elsewhere.example:8080/a", + ]) + expect(targetOf(server, requestTarget).host).toBe("127.0.0.1:3000"); + }); +}); diff --git a/e2e/pod/strictLws.ts b/e2e/pod/strictLws.ts index 2abbb2448..fa54fa146 100644 --- a/e2e/pod/strictLws.ts +++ b/e2e/pod/strictLws.ts @@ -39,6 +39,20 @@ export function ruleFor(method: string, path: string, accept: string | undefined return { send: true }; } +/** + * The URL on the server that a request's target asks for: only its path + * and query, for a request line may name a host of its own + * (`GET http://elsewhere/ HTTP/1.1`, `GET //elsewhere/`), which the proxy + * does not go to. + */ +export function targetOf(server: string, requestTarget: string): URL { + const { pathname, search } = new URL(requestTarget, server); + const target = new URL(server); + target.pathname = pathname; + target.search = search; + return target; +} + /** The headers of a response a proxy may pass on as they are (fetch has undone its encoding and length). */ const DROPPED = new Set([ "content-encoding", @@ -68,8 +82,8 @@ function headersOf(request: IncomingMessage, accept?: string): Headers { async function proxy(server: string, request: IncomingMessage, response: ServerResponse) { const method = request.method ?? "GET"; - const target = new URL(request.url ?? "/", server).href; - const rule = ruleFor(method, new URL(target).pathname, request.headers.accept); + const target = targetOf(server, request.url ?? "/"); + const rule = ruleFor(method, target.pathname, request.headers.accept); const body = await bodyOf(request); if ("refuse" in rule) { response.writeHead(rule.refuse, { "access-control-allow-origin": "*" }).end(); From 6f8f2eebea187366743894af0647522d0be5fd54 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Fri, 6 Nov 2026 13:22:24 +0100 Subject: [PATCH 09/19] fix(solid): read the LWS profile from the profile parameter alone The listing looked for the profile's URI anywhere in Content-Type, so a longer URI, or the URI in another parameter, also made the bridge ask the server whether it takes LWS. CodeQL flags the substring check. The profile parameter's list is read now and each entry compared whole. Refs: #594 --- packages/solid/src/lwsBridge.test.ts | 25 +++++++++++++++++++++++++ packages/solid/src/lwsBridge.ts | 9 ++++++++- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/packages/solid/src/lwsBridge.test.ts b/packages/solid/src/lwsBridge.test.ts index 8e9f2f4c6..afb0046dd 100644 --- a/packages/solid/src/lwsBridge.test.ts +++ b/packages/solid/src/lwsBridge.test.ts @@ -229,6 +229,31 @@ describe("lwsBridge, its listing", () => { await expect(lwsBridge(fetch)(MAIN)).rejects.toThrow("lists more than 20000"); }); + it("asks for the server's own listing only when the answer names the LWS profile as its profile", async () => { + const PROFILE = "https://www.w3.org/ns/lws/v1"; + const requestsFor = async (contentType: string | undefined) => { + const headers: Record = contentType ? { "Content-Type": contentType } : {}; + const { fetch, asked } = scripted(() => lws(200, headers)); + await lwsBridge(fetch)(MAIN); + return asked.length; + }; + for (const named of [ + `text/turtle; profile="${PROFILE}"`, + `text/turtle;PROFILE = "https://other.example/p ${PROFILE}"`, + `text/turtle; charset=utf-8; profile=${PROFILE}`, + ]) + expect(await requestsFor(named)).toBe(3); + for (const unnamed of [ + undefined, + "text/turtle", + `text/turtle; profile="${PROFILE}.elsewhere.example"`, + `text/turtle; profile=https://elsewhere.example/${PROFILE}`, + `text/turtle; note="${PROFILE}"`, + `text/turtle; profile=`, + ]) + expect(await requestsFor(unnamed)).toBe(1); + }); + it("names the listing by the address asked for when the answer has none, and passes on one of no type", async () => { const bare: typeof globalThis.fetch = async () => new Response(JSON.stringify({ items: [{ id: "a.ttl" }] }), { diff --git a/packages/solid/src/lwsBridge.ts b/packages/solid/src/lwsBridge.ts index f19a9298c..263fdcd73 100644 --- a/packages/solid/src/lwsBridge.ts +++ b/packages/solid/src/lwsBridge.ts @@ -81,6 +81,13 @@ function mediaTypeOf(contentType: string | null): string { return (contentType ?? "").split(";")[0]!.trim().toLowerCase(); } +/** Whether a Content-Type names `profile` in its `profile` parameter, a space-separated list, quoted or not. */ +function hasProfile(contentType: string | null, profile: string): boolean { + const parameter = /;\s*profile\s*=\s*(?:"([^"]*)"|([^\s;]+))/i.exec(contentType ?? ""); + const profiles = parameter?.[1] ?? parameter?.[2] ?? ""; + return profiles.split(/\s+/).some((named) => named === profile); +} + /** A response for `url`, as a server's answer is: its URL set. */ function named(response: Response, url: string): Response { Object.defineProperty(response, "url", { value: url }); @@ -123,7 +130,7 @@ async function listing( if (response.ok && mediaTypeOf(contentType) === LWS_JSON) return listingOf(fetch, url, init, response); // An LWS server that gives Turtle in the LWS profile (lws:items, no ldp:contains), or none, is asked for its own listing. - const lwsProfile = response.ok && String(contentType).includes(LWS_PROFILE); + const lwsProfile = response.ok && hasProfile(contentType, LWS_PROFILE); if (response.status !== 406 && !lwsProfile) return response; if (!(await isLws(fetch, url, init?.signal))) return response; await response.body?.cancel(); From 094063b223cb56de29ea34de2a83ee208fc0e1d4 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sat, 7 Nov 2026 01:42:37 +0100 Subject: [PATCH 10/19] build(e2e): run the CSS LWS servers on the fork's commit ef6d664f The fork answered our report with six commits: resource_metadata in its 401 challenge, a client_id that is always a URI, JSON Patch, the LWS profile, and DPoP checks at its token endpoint. The pod tests and journeys give the same results as on 311b3677, and sign-in now goes through the resource metadata. Refs: #594 --- e2e/pod/servers/css-lws/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/e2e/pod/servers/css-lws/Dockerfile b/e2e/pod/servers/css-lws/Dockerfile index 721e0a1e3..ae3712a9e 100644 --- a/e2e/pod/servers/css-lws/Dockerfile +++ b/e2e/pod/servers/css-lws/Dockerfile @@ -11,7 +11,7 @@ FROM node:22.23.4-bookworm-slim@sha256:4b150b48b4116e2a22f2c39cc27025fbf4f153e2078fd3b14d159fdb5c7279da AS build # Renovate proposes newer commits of the branch for a person to approve (renovate.json5). # renovate: datasource=git-refs depName=https://github.com/jeswr/CommunitySolidServer branch=feat/lws-next-major -ARG CSS_LWS_SHA=311b3677154256eb2f87add7dad50f1255cf50a3 +ARG CSS_LWS_SHA=ef6d664f1253662cbfcb0f1786fe45d32849e067 ARG CSS_LWS_VERSION=8.0.0-alpha.3 RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates git && rm -rf /var/lib/apt/lists/* WORKDIR /community-server From 0b71d38ae13c70547e6f95dc5e10d4927412690a Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sat, 7 Nov 2026 01:42:37 +0100 Subject: [PATCH 11/19] refactor(solid-memo-client): find the authorization server by resource metadata alone A challenge with as_uri alone was taken only for the CSS LWS fork, which left out resource_metadata. The fork now sends it on its challenges, and LWS has no as_uri, so nothing the client talks to reaches that path. It also skipped the check that the metadata names the requested URL. The tests answer as the fork does now, with the metadata route, and a challenge with as_uri alone is no LWS challenge. Refs: #594 --- docs/solid-memo-client.md | 15 +--- .../solid-memo-client/src/lwsAuth.test.ts | 71 +++++++++++-------- packages/solid-memo-client/src/lwsAuth.ts | 30 +++----- .../solid-memo-client/src/session.test.ts | 27 +++++-- 4 files changed, 73 insertions(+), 70 deletions(-) diff --git a/docs/solid-memo-client.md b/docs/solid-memo-client.md index 29248c6d4..a71b290fd 100644 --- a/docs/solid-memo-client.md +++ b/docs/solid-memo-client.md @@ -217,9 +217,9 @@ working draft: 1. **The challenge.** A request goes with the access token, as to any Solid server. A 401 is an LWS challenge when its `WWW-Authenticate` has a `Bearer` challenge with `resource_metadata` - ([RFC 9728](https://www.rfc-editor.org/rfc/rfc9728)), or with - `as_uri` ([below](#the-forks-challenge)). Its `Link` header names - the storage, with relation `https://www.w3.org/ns/lws#storage`. The + ([RFC 9728](https://www.rfc-editor.org/rfc/rfc9728)). Its `Link` + header names the storage, with relation + `https://www.w3.org/ns/lws#storage`. The requested URL must be within that storage: on its origin, its path the storage's or below it. 2. **The checks,** LWS's "Discovery Validation by a Client", before the @@ -301,15 +301,6 @@ storage behind that provider. An LWS deployment whose authorization server isn't on the identity provider's origin isn't supported yet: its storages answer 401, as to no one. -### The fork's challenge - -The Community Solid Server's LWS fork names its authorization server in -an `as_uri` parameter, and leaves out `resource_metadata`, which LWS -requires on every challenge. As a workaround, a `Bearer` challenge with -`as_uri` and no `resource_metadata` takes `as_uri` as the issuer, and -skips the resource metadata checks. The trust rule and the other -checks still hold. A challenge with both takes `resource_metadata`. - ## The public-read grant `grantPublicRead` diff --git a/packages/solid-memo-client/src/lwsAuth.test.ts b/packages/solid-memo-client/src/lwsAuth.test.ts index 22de23533..7c4f3da2e 100644 --- a/packages/solid-memo-client/src/lwsAuth.test.ts +++ b/packages/solid-memo-client/src/lwsAuth.test.ts @@ -12,12 +12,20 @@ const STORAGE = `${ISSUER}alice/`; const RESOURCE = new URL(`${STORAGE}notes/a.ttl`); const TOKEN_ENDPOINT = `${ISSUER}.well-known/lws/token`; const CONFIGURATION = `${ISSUER}.well-known/lws-configuration`; -const CHALLENGE = `Bearer as_uri="${ISSUER}", realm="${STORAGE}"`; +const METADATA = `${ISSUER}.well-known/oauth-protected-resource/alice/notes/a.ttl`; +const CHALLENGE = `Bearer resource_metadata="${METADATA}", as_uri="${ISSUER}", realm="${ISSUER}"`; const STORAGE_LINK = `<${STORAGE}>; rel="https://www.w3.org/ns/lws#storage"`; const json = (body: unknown, status = 200) => new Response(JSON.stringify(body), { status, headers: { "content-type": "application/json" } }); +/** The fork's protected resource metadata for RESOURCE, naming `issuer` as its authorization server. */ +const protectedResource = (issuer = ISSUER): Record => ({ + resource: RESOURCE.href, + authorization_servers: [issuer], + bearer_methods_supported: ["header"], +}); + /** The fork's metadata, as its /.well-known/lws-configuration serves it. */ const configuration = (): Record => ({ issuer: ISSUER, @@ -71,6 +79,7 @@ beforeEach(() => { requests = []; vi.mocked(session.idToken).mockClear(); routes = new Map([ + [METADATA, () => json(protectedResource())], [CONFIGURATION, () => json(configuration())], [STORAGE, () => json(description())], [TOKEN_ENDPOINT, exchangeAnswer], @@ -120,6 +129,7 @@ describe("LwsTokens, on the fork's answers", () => { expect( requests.map(({ url, init }) => [url, new Headers(init?.headers).get("accept")]), ).toEqual([ + [METADATA, "application/json"], [STORAGE, "application/lws+cid"], [CONFIGURATION, "application/json"], ]); @@ -145,6 +155,13 @@ describe("LwsTokens, on the fork's answers", () => { expect(issued).toBe(1); }); + it("reads the same challenge with the scope it has beside Solid", async () => { + const dual = `${CHALLENGE}, scope="openid webid"`; + await expect(discovered(undefined, unauthorized(RESOURCE, dual))).resolves.toMatchObject({ + uri: STORAGE, + }); + }); + it("knows a storage it checked by the URLs within it, the innermost first", async () => { const tokens = new LwsTokens(session); expect(tokens.storageOf(RESOURCE)).toBeUndefined(); @@ -300,14 +317,9 @@ describe("LwsTokens, on the fork's answers", () => { }); describe("LwsTokens, on LWS's resource metadata (RFC 9728)", () => { - const METADATA = `${ISSUER}.well-known/oauth-protected-resource/alice/notes/a.ttl`; const specChallenge = (metadata = METADATA) => `Bearer resource_metadata="${metadata}", error="invalid_token"`; - beforeEach(() => { - routes.set(METADATA, () => json({ resource: RESOURCE.href, authorization_servers: [ISSUER] })); - }); - it("takes the authorization server from the metadata that names the requested URI", async () => { const response = unauthorized(RESOURCE, specChallenge()); const storage = await new LwsTokens(session).discover( @@ -319,6 +331,13 @@ describe("LwsTokens, on LWS's resource metadata (RFC 9728)", () => { expect(requests[0]?.url).toBe(METADATA); }); + it("takes no authorization server from a challenge's as_uri", async () => { + const challenge = `Bearer resource_metadata="${METADATA}", as_uri="https://as.example/"`; + await expect(discovered(undefined, unauthorized(RESOURCE, challenge))).resolves.toMatchObject({ + as: { issuer: ISSUER }, + }); + }); + it.each([ ["is on another origin", "http://localhost:3989/meta", undefined], ["isn't a URL", "/relative", undefined], @@ -367,29 +386,21 @@ describe("LwsTokens, refusing", () => { ])("reads no storage description that %s past 64 KiB", async (_, said) => { routes.set(STORAGE, oversized(said)); await expect(discovered()).resolves.toBeUndefined(); - expect(requests.map(({ url }) => url)).toEqual([STORAGE]); + expect(requests.map(({ url }) => url)).toEqual([METADATA, STORAGE]); }); it("reads no resource metadata or authorization server metadata past 64 KiB", async () => { - const metadata = `${ISSUER}.well-known/oauth-protected-resource/alice/notes/a.ttl`; - routes.set( - metadata, - oversized(true, { resource: RESOURCE.href, authorization_servers: [ISSUER] }), - ); - const response = unauthorized(RESOURCE, `Bearer resource_metadata="${metadata}"`); - await expect( - new LwsTokens(session).discover(RESOURCE, response, await challengesOf(response)), - ).resolves.toBeUndefined(); + routes.set(METADATA, oversized(true, protectedResource())); + await expect(discovered()).resolves.toBeUndefined(); + routes.set(METADATA, () => json(protectedResource())); routes.set(CONFIGURATION, oversized(false, configuration())); await expect(discovered()).resolves.toBeUndefined(); }); it.each([ ["no LWS challenge", unauthorized(RESOURCE, 'DPoP error="invalid_token"')], - [ - "a Bearer challenge that names no authorization server", - unauthorized(RESOURCE, 'Bearer realm="x"'), - ], + ["a Bearer challenge without resource metadata", unauthorized(RESOURCE, 'Bearer realm="x"')], + ["a challenge with as_uri alone", unauthorized(RESOURCE, `Bearer as_uri="${ISSUER}"`)], ["a status other than 401", unauthorized(RESOURCE, CHALLENGE, STORAGE_LINK, 403)], ["no storage link", unauthorized(RESOURCE, CHALLENGE, "")], [ @@ -424,15 +435,14 @@ describe("LwsTokens, refusing", () => { }); it.each([ - ["another origin", `Bearer as_uri="https://as.example/"`], - ["no URL", `Bearer as_uri="not a URL"`], + ["another origin", "https://as.example/"], + ["no URL", "not a URL"], ])( "sends nothing to an authorization server on %s than the identity provider's", - async (_, challenge) => { - await expect( - discovered(undefined, unauthorized(RESOURCE, challenge)), - ).resolves.toBeUndefined(); - expect(requests).toEqual([]); + async (_, issuer) => { + routes.set(METADATA, () => json(protectedResource(issuer))); + await expect(discovered()).resolves.toBeUndefined(); + expect(requests.map(({ url }) => url)).toEqual([METADATA]); }, ); @@ -458,7 +468,7 @@ describe("LwsTokens, refusing", () => { ])("refuses a storage whose description %s", async (_, body) => { routes.set(STORAGE, () => json(body)); await expect(discovered()).resolves.toBeUndefined(); - expect(requests.map(({ url }) => url)).toEqual([STORAGE]); + expect(requests.map(({ url }) => url)).toEqual([METADATA, STORAGE]); }); it("takes a description whose types are lists, or full IRIs", async () => { @@ -509,9 +519,8 @@ describe("LwsTokens, refusing", () => { const issuer = `${ISSUER}tenant`; const configured = `${ISSUER}.well-known/lws-configuration/tenant`; routes.set(configured, () => json({ ...configuration(), issuer })); - await expect( - discovered(undefined, unauthorized(RESOURCE, `Bearer as_uri="${issuer}"`)), - ).resolves.toMatchObject({ as: { issuer } }); + routes.set(METADATA, () => json(protectedResource(issuer))); + await expect(discovered()).resolves.toMatchObject({ as: { issuer } }); expect(requests.map(({ url }) => url)).toContain(configured); }); }); diff --git a/packages/solid-memo-client/src/lwsAuth.ts b/packages/solid-memo-client/src/lwsAuth.ts index ea2687200..7fe14bc1f 100644 --- a/packages/solid-memo-client/src/lwsAuth.ts +++ b/packages/solid-memo-client/src/lwsAuth.ts @@ -27,12 +27,6 @@ import { clientAuth, requestOptions, secureEndpoint, timeLimited } from "./oidcL * server that names its own authorization server never gets it. * - A storage's token goes only to URLs within that storage (same origin, * and the storage's path as a prefix), never elsewhere on its server. - * - * Deviation, for the Community Solid Server LWS fork: its challenge - * names the authorization server in `as_uri` and has no - * `resource_metadata` (RFC 9728), which LWS requires. A challenge with - * `as_uri` alone takes it as the authorization server's issuer; the - * trust rule above holds it to the identity provider's origin. */ /** The Link relation naming the storage a resource is in. */ @@ -141,18 +135,16 @@ function abortable(work: Promise, signal: AbortSignal | null | undefined): } /** - * The authorization server's issuer the challenge leads to. With - * `resource_metadata` (LWS): the metadata is on the requested URI's - * scheme and authority, names that URI as its `resource`, and its first - * authorization server is the one. Without it, the fork's `as_uri`. + * The authorization server's issuer the challenge's `resource_metadata` + * (RFC 9728) leads to: the metadata is on the requested URI's scheme and + * authority, names that URI as its `resource`, and its first + * authorization server is the one. */ async function issuerOf( url: URL, - challenge: oauth.WWWAuthenticateChallenge, + metadataUri: string, signal: AbortSignal | null | undefined, ): Promise { - const { resource_metadata: metadataUri, as_uri: asUri } = challenge.parameters; - if (metadataUri === undefined) return asUri; const metadataUrl = URL.parse(metadataUri); if (metadataUrl?.origin !== url.origin) return undefined; const metadata = await getJson(metadataUrl.href, "application/json", signal); @@ -250,17 +242,15 @@ export class LwsTokens { challenges: oauth.WWWAuthenticateChallenge[], signal?: AbortSignal | null, ): Promise { - const challenge = challenges.find( - ({ scheme, parameters }) => - scheme === "bearer" && - (parameters.resource_metadata !== undefined || parameters.as_uri !== undefined), - ); - if (response.status !== 401 || challenge === undefined) return undefined; + const metadataUri = challenges.find( + ({ scheme, parameters }) => scheme === "bearer" && parameters.resource_metadata !== undefined, + )?.parameters.resource_metadata; + if (response.status !== 401 || metadataUri === undefined) return undefined; const uri = linkTargets(response, LWS_STORAGE, url.href).find((target) => within(url, target)); if (uri === undefined) return undefined; const known = this.#storages.get(uri); if (known !== undefined) return known; - const issuer = URL.parse((await issuerOf(url, challenge, signal)) ?? ""); + const issuer = URL.parse((await issuerOf(url, metadataUri, signal)) ?? ""); if ( issuer?.origin !== new URL(this.#session.issuer).origin || !(await describesStorage(uri, url, signal)) diff --git a/packages/solid-memo-client/src/session.test.ts b/packages/solid-memo-client/src/session.test.ts index c2751ad64..e0e6aac67 100644 --- a/packages/solid-memo-client/src/session.test.ts +++ b/packages/solid-memo-client/src/session.test.ts @@ -226,6 +226,7 @@ async function idToken(claims: Record): Promise { /** A Linked Web Storage storage, its authorization server the identity provider's (lwsAuth.ts). */ const STORAGE = "https://pod.example/alice/"; +const POD_METADATA = "https://pod.example/.well-known/oauth-protected-resource"; const LWS_TOKEN_ENDPOINT = `${ISSUER}.well-known/lws/token`; /** The LWS authorization server on the provider's origin, as the Community Solid Server LWS fork runs one. */ @@ -259,14 +260,19 @@ function exchangeEndpoint(init: RequestInit | undefined): Response { } /** - * The storage as the fork serves it: its description, a request with - * one of its tokens within it, and a 401 with its challenge to anything - * else, a DPoP request among them, which it reads as no one's. + * The storage as the fork serves it: its protected resource metadata + * and description, a request with one of its tokens within it, and a + * 401 with its challenge to anything else, a DPoP request among them, + * which it reads as no one's. */ function lwsPod(asUri = ISSUER) { return vi.fn(async (input, init) => { const url = input instanceof Request ? input.url : String(input); const headers = new Headers(init?.headers); + if (url.startsWith(`${POD_METADATA}/`)) { + const resource = new URL(url.slice(POD_METADATA.length), STORAGE).href; + return json({ resource, authorization_servers: [asUri] }); + } if (url === STORAGE && headers.get("accept") === "application/lws+cid") { return json({ "@context": ["https://www.w3.org/ns/cid/v1", "https://www.w3.org/ns/lws/v1"], @@ -283,7 +289,7 @@ function lwsPod(asUri = ISSUER) { return new Response(null, { status: 401, headers: { - "www-authenticate": `Bearer as_uri="${asUri}", realm="${STORAGE}"${bearer ? ', error="invalid_token"' : ""}`, + "www-authenticate": `Bearer resource_metadata="${POD_METADATA}${new URL(url).pathname}", as_uri="${asUri}", realm="${STORAGE}"${bearer ? ', error="invalid_token"' : ""}`, link: `<${STORAGE}>; rel="https://www.w3.org/ns/lws#storage"`, }, }); @@ -1293,10 +1299,14 @@ describe("createSessionClient, on a Linked Web Storage storage", () => { return client; } - /** The requests the pod was sent, as their URL and Authorization; not the description's. */ + /** The requests the pod was sent, as their URL and Authorization; not the checks', for its metadata and description. */ const sent = (pod: ReturnType) => pod.mock.calls - .filter(([, init]) => new Headers(init?.headers).get("accept") !== "application/lws+cid") + .filter( + ([input, init]) => + !String(input).startsWith(POD_METADATA) && + new Headers(init?.headers).get("accept") !== "application/lws+cid", + ) .map(([input, init]) => [ input instanceof Request ? input.url : String(input), new Headers(init?.headers).get("authorization"), @@ -1471,7 +1481,10 @@ describe("createSessionClient, on a Linked Web Storage storage", () => { provider.pod = pod; await expect(client.fetch(RESOURCE)).resolves.toMatchObject({ status: 401 }); expect(lws.exchanges).toEqual([]); - expect(pod.mock.calls.map(([input]) => String(input))).toEqual([RESOURCE]); + expect(pod.mock.calls.map(([input]) => String(input))).toEqual([ + RESOURCE, + `${POD_METADATA}/alice/notes/a.ttl`, + ]); }); it("forgets the storages' tokens at logout", async () => { From 230e6891decc16e8bd6b5fd175e0ba7fe7210d0f Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sat, 7 Nov 2026 03:30:57 +0100 Subject: [PATCH 12/19] test(e2e): sign in on the CSS LWS servers The signed-in tests left out css-lws and css-lws-only, so CI never signed in to an LWS storage: a change in the fork's challenge, metadata or token exchange would pass, and show only when someone ran the journeys by hand. Both now have an instance that isn't open, on the fork's own configuration. The harness scripts the Community Solid Server's login for an ID token, which client credentials don't give, and the requests go through the app's own session from solid-memo-client, the only client that makes the exchange, so e2e-pod may import it in one file. Beside the shared tests, one checks which way the client signs in on each server, and one that a Bearer token holds to its storage and its user's access. Refs: #594 --- docs/boundaries.md | 4 +- docs/lws.md | 5 +- docs/testing.md | 21 ++- e2e/pod/README.md | 5 +- e2e/pod/package.json | 1 + e2e/pod/servers.ts | 2 + e2e/pod/servers/css-lws-only/signed-in.yml | 19 +++ e2e/pod/servers/css-lws/signed-in.yml | 19 +++ e2e/pod/signedIn.ts | 165 ++++++++++++++++++--- e2e/pod/signedInClient.ts | 52 +++++++ e2e/pod/src/signedIn.integration.test.ts | 72 ++++++++- package-lock.json | 1 + scripts/checkBoundaries.ts | 6 +- 13 files changed, 333 insertions(+), 39 deletions(-) create mode 100644 e2e/pod/servers/css-lws-only/signed-in.yml create mode 100644 e2e/pod/servers/css-lws/signed-in.yml create mode 100644 e2e/pod/signedInClient.ts diff --git a/docs/boundaries.md b/docs/boundaries.md index 77c95ebce..63a4ad858 100644 --- a/docs/boundaries.md +++ b/docs/boundaries.md @@ -88,8 +88,8 @@ boundaries." | `composition` | `application`, `domain`, `vocab`, `solid`, `browser` | | | `web` | `application`, `domain`, `vocab`, `ui`, `composition`, `studio`, `site` | `composition`, and `studio` with a dynamic `import()`: `src/main.tsx` only; `site`: `vite.config.ts` (the site's build, which renders the docs) and tests only | | `studio` | `application`, `domain`, `vocab`, `ui` | | -| `e2e-pod` | `application`, `domain`, `vocab`, `solid` | | -| `solid-memo-client` | `solid` | No package imports it: the build puts its modules in place of `solid`'s client modules ([below](#the-solid-client-libraries)) | +| `e2e-pod` | `application`, `domain`, `vocab`, `solid`, `solid-memo-client` | `solid-memo-client`: `signedInClient.ts` only, which signs in to a Linked Web Storage server | +| `solid-memo-client` | `solid` | No package the app is made of imports it: the build puts its modules in place of `solid`'s client modules ([below](#the-solid-client-libraries)) | | `e2e-journeys` | `ui`, `studio` | `ui` and `studio`: `harness/strings.ts` only, for their messages | Browser code is the `src/` of `markdown`, `vocab`, `domain`, diff --git a/docs/lws.md b/docs/lws.md index 704eaaacd..9b0a12855 100644 --- a/docs/lws.md +++ b/docs/lws.md @@ -119,8 +119,9 @@ app the ID token only in the address that ends the session, so a build that sign ## Trying it by hand -The end-to-end tests send no credentials, so sign-in on an LWS storage -is tried by hand, with the user journeys +[The signed-in tests](testing.md#the-signed-in-tests) sign in to an LWS +storage without a browser. The login in a browser is tried by hand, with +the user journeys ([e2e/journeys/css-lws/compose.yml](../e2e/journeys/css-lws/compose.yml)). The fork's image, which `npm run servers -w @solid-memo/e2e-pod -- css-lws` builds, runs its own `config/lws.json` behind the journeys' diff --git a/docs/testing.md b/docs/testing.md index 29ebcb97c..ac323450e 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -111,9 +111,8 @@ on the same 8.0.0-alpha.3 as css-8, so a failure only css-lws shows comes from the LWS changes. Its configuration is the fork's `config/solid-lws.json` in memory, with the root as the storage. css-lws-only runs the same image with the fork's `config/lws.json` -instead: LWS alone. The tests send no credentials, so neither shows -how the app signs in to an LWS storage; [lws.md](lws.md) says how -that's tried by hand. The image is built from code nobody has reviewed, on CI's runners and on the +instead: LWS alone. The app's sign-in to an LWS storage is shown by +[the signed-in tests](#the-signed-in-tests). The image is built from code nobody has reviewed, on CI's runners and on the machine of anyone who runs `SOLID_SERVERS=all`. Its dependencies' install scripts don't run, and its final stage keeps only what the server runs. GitHub's dependency review doesn't see the lockfile it @@ -372,6 +371,7 @@ request. | `css-6` | Client credentials from its older identity API, DPoP-bound | | `nss-6`, `nss-5` | The authorization-code flow, its login and consent forms filled in by script; each pod on a subdomain of localhost | | `jss` | Its own email and password endpoint, `/idp/credentials`, DPoP-bound | +| `css-lws`, `css-lws-only` | The authorization-code flow, the account API answering for its login and consent pages, for an ID token as well as a DPoP-bound access token | `nextcloud` is left out. Its login is Nextcloud's browser form and the Solid app's approval page, which nothing here scripts. @@ -390,6 +390,21 @@ The tests check that: - an invalid token gets a 401, the document stays as it was, and the app ends the session only if the answer names the token invalid. +On `css-lws` and `css-lws-only`, the sign-in client is the app's own +([signedInClient.ts](../e2e/pod/signedInClient.ts)): `solid-memo-client`'s +session, below the bridge, as the app wires it, since Inrupt's client +gives the app no ID token. So the requests are the ones a browser's +session makes, and: + +- on LWS alone, a DPoP request is answered with the LWS challenge, and + the client goes through the resource metadata and the exchange to a + Bearer token; beside Solid, it stays on DPoP; +- a Bearer token from the exchange works for its user's access, and only + in the storage it's for. + +There an invalid token is the ID token too: on LWS alone a bad access +token is no one's, and it's the exchange that refuses. + The app has no way to take access back, so nothing tests removing it. A real expired token isn't tested either: the shortest lifetime on these servers is ten minutes, too long to wait for in a test. A server checks diff --git a/e2e/pod/README.md b/e2e/pod/README.md index e14b7e8ec..e8a694f80 100644 --- a/e2e/pod/README.md +++ b/e2e/pod/README.md @@ -8,7 +8,7 @@ Server, the JavaScript Solid Server, Nextcloud and others, each in ## What is in it - `src/*.integration.test.ts`: the tests, run against every server. -- `src/signedIn.integration.test.ts`, `signedIn.ts`: the signed-in tests, +- `src/signedIn.integration.test.ts`, `signedIn.ts`, `signedInClient.ts`: the signed-in tests, on each server's instance that isn't open (`signed-in.yml`), and how their accounts get a token ([testing.md](../../docs/testing.md#the-signed-in-tests)). - `servers/`: one folder per server, with its compose file and, where @@ -24,7 +24,8 @@ It exports nothing; no package may import it. ## Boundaries -It may import `application`, `domain`, `vocab` and `solid`, and `n3` +It may import `application`, `domain`, `vocab` and `solid`, +`solid-memo-client` in `signedInClient.ts`, and `n3` for the server contract and the tests ([boundaries.md](../../docs/boundaries.md)). diff --git a/e2e/pod/package.json b/e2e/pod/package.json index 0d8c3465d..760404d9c 100644 --- a/e2e/pod/package.json +++ b/e2e/pod/package.json @@ -20,6 +20,7 @@ "@solid-memo/application": "*", "@solid-memo/domain": "*", "@solid-memo/solid": "*", + "@solid-memo/solid-memo-client": "*", "@solid-memo/vocab": "*" }, "devDependencies": { diff --git a/e2e/pod/servers.ts b/e2e/pod/servers.ts index 70644d92d..d0431446a 100644 --- a/e2e/pod/servers.ts +++ b/e2e/pod/servers.ts @@ -187,6 +187,7 @@ export const SERVERS = { // The fork's commit, on the release css-8 runs, so a failure only it shows comes from its LWS changes. version: () => `${dockerfileArg("css-lws", "CSS_LWS_VERSION")} (${dockerfileArg("css-lws", "CSS_LWS_SHA").slice(0, 8)})`, + signedIn: { login: "css-lws", host: "127.0.0.1" }, }, "css-lws-only": { label: "Community Solid Server with LWS only", @@ -198,6 +199,7 @@ export const SERVERS = { // css-lws's image, run with LWS alone: what the app does where Solid's own requests are not taken (docs/lws.md). version: () => `${dockerfileArg("css-lws", "CSS_LWS_VERSION")} (${dockerfileArg("css-lws", "CSS_LWS_SHA").slice(0, 8)})`, + signedIn: { login: "css-lws", host: "127.0.0.1" }, }, } satisfies Record; diff --git a/e2e/pod/servers/css-lws-only/signed-in.yml b/e2e/pod/servers/css-lws-only/signed-in.yml new file mode 100644 index 000000000..cba547309 --- /dev/null +++ b/e2e/pod/servers/css-lws-only/signed-in.yml @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +# +# SPDX-License-Identifier: MIT + +# The same server, not open: the fork's own configuration, in files on a tmpfs, each account's pod a storage private to it (docs/testing.md). +services: + tls: + extends: { file: ../compose.tls.yml, service: tls } + environment: { E2E_UPSTREAM: "3000" } + server: + extends: { file: compose.yml, service: server } + command: !override ["-c", "config/lws.json", "-f", "/data", "-b", "https://127.0.0.1:${E2E_PORT:?}/", "-l", "warn"] + ports: !reset [] + network_mode: "service:tls" + depends_on: [tls] + # Caddy's CA is made as it starts, and this server is thrown away after + # the run: it trusts whatever certificate its own requests meet. + environment: { NODE_TLS_REJECT_UNAUTHORIZED: "0" } + tmpfs: ["/data:uid=1000,gid=1000,mode=0700"] diff --git a/e2e/pod/servers/css-lws/signed-in.yml b/e2e/pod/servers/css-lws/signed-in.yml new file mode 100644 index 000000000..c5e8343fc --- /dev/null +++ b/e2e/pod/servers/css-lws/signed-in.yml @@ -0,0 +1,19 @@ +# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +# +# SPDX-License-Identifier: MIT + +# The same server, not open: the fork's own configuration, in files on a tmpfs, each account's pod a storage private to it (docs/testing.md). +services: + tls: + extends: { file: ../compose.tls.yml, service: tls } + environment: { E2E_UPSTREAM: "3000" } + server: + extends: { file: compose.yml, service: server } + command: !override ["-c", "config/solid-lws.json", "-f", "/data", "-b", "https://127.0.0.1:${E2E_PORT:?}/", "-l", "warn"] + ports: !reset [] + network_mode: "service:tls" + depends_on: [tls] + # Caddy's CA is made as it starts, and this server is thrown away after + # the run: it trusts whatever certificate its own requests meet. + environment: { NODE_TLS_REJECT_UNAUTHORIZED: "0" } + tmpfs: ["/data:uid=1000,gid=1000,mode=0700"] diff --git a/e2e/pod/signedIn.ts b/e2e/pod/signedIn.ts index 593701362..646400e2d 100644 --- a/e2e/pod/signedIn.ts +++ b/e2e/pod/signedIn.ts @@ -15,7 +15,7 @@ import type { SessionClient } from "@solid-memo/solid/client/session"; */ /** How a server makes accounts and gives out tokens. */ -export type Login = "css" | "css-6" | "nss" | "jss"; +export type Login = "css" | "css-6" | "css-lws" | "nss" | "jss"; export interface Account { webId: string; @@ -48,6 +48,8 @@ export interface Session { issuer: string; accessToken: string; key: DpopKey; + /** The ID token and client of a login that gives them, which a Linked Web Storage server exchanges (cssLogin). */ + openId?: { idToken: string; clientId: string; expiresAt: number }; } export interface DpopKey { @@ -261,6 +263,7 @@ async function nssAccount(server: string, username: string): Promise { const ACCOUNT_MAKERS = { css: cssAccount, "css-6": css6Account, + "css-lws": cssAccount, nss: nssAccount, jss: jssAccount, } satisfies Record Promise>; @@ -319,6 +322,30 @@ function redirectOf(what: string, response: Response, base: string): URL { return new URL(location, base); } +/** Requests as a browser's, keeping cookies and following no redirect: a form's body if it is given one. */ +function browser() { + const cookies = new Map(); + return async (url: string | URL, form?: Record, json?: unknown) => { + const response = await fetch(url, { + method: form === undefined && json === undefined ? "GET" : "POST", + redirect: "manual", + headers: { + accept: json === undefined ? "text/html" : "application/json", + cookie: [...cookies].map(([name, value]) => `${name}=${value}`).join("; "), + ...(form !== undefined && FORM_BODY), + ...(json !== undefined && JSON_BODY), + }, + ...(form !== undefined && { body: new URLSearchParams(form) }), + ...(json !== undefined && { body: JSON.stringify(json) }), + }); + for (const cookie of response.headers.getSetCookie()) { + const [name, ...value] = cookie.split(";")[0]!.split("="); + cookies.set(name!, value.join("=")); + } + return response; + }; +} + /** * A token through node-solid-server's authorization-code flow, its forms * filled in as a browser would (it has no other way to one): a client @@ -344,24 +371,7 @@ async function codeFlowToken(server: string, account: Account, key: DpopKey): Pr state: randomUUID(), nonce: randomUUID(), }; - const cookies = new Map(); - const browse = async (url: string | URL, form?: Record) => { - const response = await fetch(url, { - method: form === undefined ? "GET" : "POST", - redirect: "manual", - headers: { - accept: "text/html", - cookie: [...cookies].map(([name, value]) => `${name}=${value}`).join("; "), - ...(form !== undefined && FORM_BODY), - }, - ...(form !== undefined && { body: new URLSearchParams(form) }), - }); - for (const cookie of response.headers.getSetCookie()) { - const [name, ...value] = cookie.split(";")[0]!.split("="); - cookies.set(name!, value.join("=")); - } - return response; - }; + const browse = browser(); await browse(`${server}authorize?${new URLSearchParams(params).toString()}`); let next = redirectOf( "Logging in", @@ -404,16 +414,127 @@ async function codeFlowToken(server: string, account: Account, key: DpopKey): Pr return access_token; } +/** + * The tokens of a login on the Community Solid Server's authorization-code + * flow, its account API answering as its pages would: a client registered, + * the login, the WebID picked, the consent, the code redeemed. Client + * credentials give no ID token, which the Linked Web Storage exchange takes. + */ +async function cssLogin( + server: string, + account: Account, + key: DpopKey, +): Promise> { + const redirectUri = "https://signed-in.example/callback"; + const provider = await answered<{ + registration_endpoint: string; + authorization_endpoint: string; + token_endpoint: string; + }>("Reading the provider", await fetch(`${server}.well-known/openid-configuration`)); + const { client_id: clientId } = await answered<{ client_id: string }>( + "Registering a client", + await post(provider.registration_endpoint, { + redirect_uris: [redirectUri], + response_types: ["code"], + grant_types: ["authorization_code"], + token_endpoint_auth_method: "none", + application_type: "web", + scope: "openid webid", + }), + ); + const verifier = randomUUID() + randomUUID(); + const browse = browser(); + const follow = async (start: string) => { + let next = new URL(start, server); + for (let step = 0; step < 8; step++) { + if (next.href.startsWith(redirectUri) || next.pathname.startsWith("/.account/")) return next; + next = redirectOf(`Following the login at ${next.pathname}`, await browse(next), server); + } + throw new Error(`The login never came back, ending at ${next.href}.`); + }; + const step = async (path: string, body: unknown) => + follow( + ( + await answered<{ location: string }>( + `Answering ${path}`, + await browse(`${server}.account/${path}`, undefined, body), + ) + ).location, + ); + const authorize = new URL(provider.authorization_endpoint); + authorize.search = new URLSearchParams({ + response_type: "code", + client_id: clientId, + redirect_uri: redirectUri, + scope: "openid webid", + state: randomUUID(), + nonce: randomUUID(), + code_challenge: createHash("sha256").update(verifier).digest("base64url"), + code_challenge_method: "S256", + }).toString(); + await follow(authorize.href); + await step("login/password/", { email: account.email, password: account.password }); + await step("oidc/pick-webid/", { webId: account.webId }); + const back = await step("oidc/consent/", { remember: false }); + const { access_token: accessToken, id_token: idToken } = await answered<{ + access_token: string; + id_token: string; + }>( + "Redeeming the code", + await fetch(provider.token_endpoint, { + method: "POST", + headers: { ...FORM_BODY, dpop: dpopProof(key, "POST", provider.token_endpoint) }, + body: new URLSearchParams({ + grant_type: "authorization_code", + client_id: clientId, + code: back.searchParams.get("code") ?? "", + redirect_uri: redirectUri, + code_verifier: verifier, + }), + }), + ); + const { exp } = JSON.parse(Buffer.from(idToken.split(".")[1]!, "base64url").toString()) as { + exp: number; + }; + return { accessToken, openId: { idToken, clientId, expiresAt: exp * 1000 } }; +} + const TOKEN_MAKERS = { css: clientCredentialsToken, "css-6": clientCredentialsToken, nss: codeFlowToken, jss: passwordToken, -} satisfies Record Promise>; +} satisfies Record< + Exclude, + (server: string, account: Account, key: DpopKey) => Promise +>; /** The account signed in, on a key of its own. */ export async function signIn(server: SignedInServer, account: Account): Promise { const key = newDpopKey(); - const accessToken = await TOKEN_MAKERS[server.login](server.url, account, key); - return { webId: account.webId, issuer: server.url, accessToken, key }; + const signedIn = { webId: account.webId, issuer: server.url, key }; + if (server.login === "css-lws") + return { ...signedIn, ...(await cssLogin(server.url, account, key)) }; + return { + ...signedIn, + accessToken: await TOKEN_MAKERS[server.login](server.url, account, key), + }; +} + +/** The Bearer token the session's ID token is exchanged for, for `storage`, as the app's client asks for it (lwsAuth.ts). */ +export async function exchangedToken(session: Session, storage: string): Promise { + const { access_token } = await answered<{ access_token: string }>( + "Exchanging the ID token", + await fetch(`${session.issuer}.well-known/lws/token`, { + method: "POST", + headers: FORM_BODY, + body: new URLSearchParams({ + grant_type: "urn:ietf:params:oauth:grant-type:token-exchange", + resource: storage, + subject_token: session.openId!.idToken, + subject_token_type: "urn:ietf:params:oauth:token-type:id_token", + }), + }), + ); + return access_token; } diff --git a/e2e/pod/signedInClient.ts b/e2e/pod/signedInClient.ts new file mode 100644 index 000000000..f82e52659 --- /dev/null +++ b/e2e/pod/signedInClient.ts @@ -0,0 +1,52 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { createPublicKey } from "node:crypto"; +import { OidcSession, setCurrentSession } from "@solid-memo/solid-memo-client/oidcSession"; +import { lwsBridge } from "@solid-memo/solid/lwsBridge"; +import type { SessionClient } from "@solid-memo/solid/client/session"; +import { sessionClient, type Session } from "./signedIn.ts"; + +const ECDSA = { name: "ECDSA", namedCurve: "P-256" }; + +/** + * The sign-in client over a session. Where it has an ID token, that of + * the app's own Solid client, whose session makes a Linked Web Storage + * server's token exchange (docs/lws.md), with the app's bridge above it, + * as appUseCases.ts wires it. It is apart from signedIn.ts because + * servers.ts, which node runs as it is, imports that. + */ +export function clientOf(session: Session): SessionClient { + const plain = sessionClient(session); + if (session.openId === undefined) return plain; + const { accessToken, webId, issuer, key, openId } = session; + // It reads each request's URL against the page's. + const page = globalThis as { window?: unknown }; + page.window ??= Object.assign(new EventTarget(), { location: { href: issuer } }); + const oidc = new OidcSession( + { issuer }, + { + webId, + client: { client_id: openId.clientId, token_endpoint_auth_method: "none" }, + keys: { + privateKey: key.privateKey.toCryptoKey(ECDSA, false, ["sign"]), + publicKey: createPublicKey(key.privateKey).toCryptoKey(ECDSA, true, ["verify"]), + } as CryptoKeyPair, + }, + { + accessToken, + expiresAt: Number.POSITIVE_INFINITY, + idToken: openId.idToken, + idTokenExpiresAt: openId.expiresAt, + }, + ); + return { + ...plain, + fetch: lwsBridge((input, init) => { + // The app has one user at a time; these tests have two, one request at a time. + setCurrentSession(oidc); + return oidc.fetch(input, init); + }), + }; +} diff --git a/e2e/pod/src/signedIn.integration.test.ts b/e2e/pod/src/signedIn.integration.test.ts index d28232417..33638872a 100644 --- a/e2e/pod/src/signedIn.integration.test.ts +++ b/e2e/pod/src/signedIn.integration.test.ts @@ -19,9 +19,10 @@ import { createAuthFetch, SESSION_EXPIRED_EVENT } from "@solid-memo/solid/authFe import { grantPublicRead } from "@solid-memo/solid/client/access"; import { createPodFence } from "@solid-memo/solid/podFence"; import { createSolidReleasePublisher } from "@solid-memo/solid/solidReleasePublisher"; +import { clientOf } from "../signedInClient.ts"; import { + exchangedToken, resolveSubdomainsOfLocalhost, - sessionClient, signIn, trustCa, type Session, @@ -36,7 +37,7 @@ const ORIGINAL = `<#entry> "first words" .`; /** The app's fetch for a session, its fence around it, as createAppUseCases wires it. */ function appFetch(session: Session, pod: string): typeof globalThis.fetch { return createPodFence( - createAuthFetch(sessionClient(session), { ownStorages: async () => [pod] }).fetch, + createAuthFetch(clientOf(session), { ownStorages: async () => [pod] }).fetch, ); } @@ -85,7 +86,7 @@ describe.each(SERVERS)("signed in on $name", (server) => { ownerSession = await signIn(server, owner); ownerFetch = appFetch(ownerSession, owner.pod); // No fence: what refuses the stranger must be the server, whatever they send it. - strangerFetch = createAuthFetch(sessionClient(await signIn(server, stranger)), { + strangerFetch = createAuthFetch(clientOf(await signIn(server, stranger)), { ownStorages: async () => [stranger.pod], }).fetch; others = [ @@ -159,7 +160,7 @@ describe.each(SERVERS)("signed in on $name", (server) => { /** The release publisher as createAppUseCases wires it, as the session's user. */ function publisherOf(session: Session, pod: string) { - const fetch = createAuthFetch(sessionClient(session), { + const fetch = createAuthFetch(clientOf(session), { ownStorages: async () => [pod], }).fetch; return createSolidReleasePublisher({ @@ -257,12 +258,19 @@ describe.each(SERVERS)("signed in on $name", (server) => { it(`answers ${what} with a 401 the app can read, the document staying as it was`, async () => { const url = await privateDocument(); const before = await contents(url); - const window = new EventTarget(); + const window = Object.assign(new EventTarget(), { location: { href: server.url } }); const expired = vi.fn(); window.addEventListener(SESSION_EXPIRED_EVENT, expired); vi.stubGlobal("window", window); const spoiled = appFetch( - { ...ownerSession, accessToken: spoil(ownerSession.accessToken) }, + { + ...ownerSession, + accessToken: spoil(ownerSession.accessToken), + // On Linked Web Storage alone a bad access token is no one's, and the ID token is what's exchanged. + ...(ownerSession.openId && { + openId: { ...ownerSession.openId, idToken: spoil(ownerSession.openId.idToken) }, + }), + }, owner.pod, ); const read = await spoiled(url); @@ -276,4 +284,56 @@ describe.each(SERVERS)("signed in on $name", (server) => { expect(await contents(url)).toBe(before); }); } + + /** The rest is Linked Web Storage's (docs/lws.md): a token for a storage, from the ID token. */ + if (server.login !== "css-lws") return; + + it("signs in with the app's client as the server takes it: the exchange on LWS alone, and DPoP beside Solid", async () => { + const url = await privateDocument(); + const fresh = appFetch(await signIn(server, owner), owner.pod); + const sent = vi.spyOn(globalThis, "fetch"); + try { + expect(await statusOf(fresh, url)).toBe(200); + const requests = sent.mock.calls.map(([input, init]) => ({ + url: String(input), + scheme: new Headers(init?.headers).get("authorization")?.split(" ")[0], + })); + const exchanges = requests.filter((request) => request.url.endsWith("/lws/token")); + expect(requests[0]).toEqual({ url, scheme: "DPoP" }); + if (server.id === "css-lws-only") { + expect(exchanges).toHaveLength(1); + expect(requests.some((request) => request.url.includes("oauth-protected-resource"))).toBe( + true, + ); + expect(requests.at(-1)).toEqual({ url, scheme: "Bearer" }); + } else { + expect(exchanges).toEqual([]); + expect(requests).toEqual([{ url, scheme: "DPoP" }]); + } + } finally { + sent.mockRestore(); + } + }); + + it("holds a Bearer token from the exchange to its storage and its user's access", async () => { + const url = await privateDocument(); + const bearer = (token: string) => ({ headers: { authorization: `Bearer ${token}` } }); + const ownerToken = await exchangedToken(ownerSession, owner.pod); + expect(await statusOf(globalThis.fetch, url, bearer(ownerToken))).toBe(200); + const theirs = new URL(`private-${randomUUID()}/diary.ttl`, stranger.pod).href; + const written = await strangerFetch(theirs, { method: "PUT", headers: TURTLE, body: ORIGINAL }); + expect(written.status).toBe(201); + expect(await statusOf(globalThis.fetch, theirs, bearer(ownerToken))).toBe(401); + const strangerSession = await signIn(server, stranger); + const strangerToken = await exchangedToken(strangerSession, owner.pod); + expect(await statusOf(globalThis.fetch, url, bearer(strangerToken))).toBe(403); + const before = await contents(url); + expect( + await statusOf(globalThis.fetch, url, { + ...CHANGES[0]!.init, + headers: { ...TURTLE, ...bearer(strangerToken).headers }, + }), + ).toBe(403); + expect(await contents(url)).toBe(before); + }); }); diff --git a/package-lock.json b/package-lock.json index f0ec5a9b6..bbd788f2a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -103,6 +103,7 @@ "@solid-memo/application": "*", "@solid-memo/domain": "*", "@solid-memo/solid": "*", + "@solid-memo/solid-memo-client": "*", "@solid-memo/vocab": "*" }, "devDependencies": { diff --git a/scripts/checkBoundaries.ts b/scripts/checkBoundaries.ts index 622f99fce..f6fad76f8 100644 --- a/scripts/checkBoundaries.ts +++ b/scripts/checkBoundaries.ts @@ -51,9 +51,9 @@ export const LAYERS: Record = { web: ["application", "domain", "vocab", "ui", "composition", "studio", "site"], // The Studio, a second app on the same layers as Solid Memo's, which the site's page loads. studio: ["application", "domain", "vocab", "ui"], - "e2e-pod": ["application", "domain", "vocab", "solid"], + "e2e-pod": ["application", "domain", "vocab", "solid", "solid-memo-client"], // A second implementation of solid's client modules, which it tests with solid's contract. No package - // imports it: the build puts it in their place when SOLID_CLIENT names it (scripts/solidClient.ts). + // the app is made of imports it: the build puts it in their place when SOLID_CLIENT names it (scripts/solidClient.ts). "solid-memo-client": ["solid"], // The journeys drive the built app in a browser; they read only its text. "e2e-journeys": ["ui", "studio"], @@ -76,6 +76,8 @@ export const ONLY_FROM: Record> = { shacl: { turtle: /^node\//, markdown: /^node\// }, // The app's messages (packages/ui/src/i18n, apps/studio/src/i18n), which the page objects find text by. "e2e-journeys": { ui: /^harness\/strings\.ts$/, studio: /^harness\/strings\.ts$/ }, + // The signed-in tests sign in to a Linked Web Storage server with the only client that can. + "e2e-pod": { "solid-memo-client": /^signedInClient\.ts$/ }, }; /** Within a package, the entry points of an allowed package that alone it may import. */ From f88b64444a610f4250d96112a55eb2e02bbe05f5 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sat, 7 Nov 2026 12:24:42 +0100 Subject: [PATCH 13/19] refactor(solid): remove the Linked Web Storage bridge The bridge made the app's Solid requests the way LWS makes them, but only for a server that refuses Solid requests. No server the app is tested on does: css-lws-only ran the full pod suite, signed in too, with the bridge as a pass-through. Only unit tests against a fake pod and a proxy run by hand reached its fallbacks, and CI ran neither. Even on such a server it would have helped only part of the way. LWS names a new resource itself, and the bridge refused a creation the server named otherwise, since every address in an instance is the app's to choose. That left about 400 lines that write and delete on every pod request in the shipped app, with nothing exercising them. LWS storage discovery, Turtle-first link reads and the LWS sign-in stay. docs/lws.md now says the app needs a server that takes the Solid Protocol's requests too. The bridge is in the history if a server needs it. Refs: #594 --- docs/docs.ttl | 2 +- docs/lws.md | 69 +--- docs/standards.md | 2 +- docs/testing.md | 15 +- e2e/pod/contract.ts | 10 +- e2e/pod/globalSetup.ts | 4 +- e2e/pod/package.json | 3 +- e2e/pod/setup.ts | 37 +- e2e/pod/signedInClient.ts | 8 +- e2e/pod/strictLws.test.ts | 53 --- e2e/pod/strictLws.ts | 128 ------- packages/composition/src/appUseCases.ts | 9 +- packages/solid/src/lwsBridge.test.ts | 468 ------------------------ packages/solid/src/lwsBridge.ts | 404 -------------------- packages/solid/src/testing/lwsPod.ts | 192 ---------- 15 files changed, 31 insertions(+), 1373 deletions(-) delete mode 100644 e2e/pod/strictLws.test.ts delete mode 100644 e2e/pod/strictLws.ts delete mode 100644 packages/solid/src/lwsBridge.test.ts delete mode 100644 packages/solid/src/lwsBridge.ts delete mode 100644 packages/solid/src/testing/lwsPod.ts diff --git a/docs/docs.ttl b/docs/docs.ttl index 82f1d43d7..a7fc7b96c 100644 --- a/docs/docs.ttl +++ b/docs/docs.ttl @@ -82,7 +82,7 @@ schema:isPartOf <#the-code> ; schema:position 46 ; schema:learningResourceType "explanation" ; - dcterms:abstract "how the app works on a Linked Web Storage server: finding the storage, the bridge that makes the app's Solid requests as LWS has them, signing in with a token exchange, and what LWS can't give the app yet."@en . + dcterms:abstract "how the app works on a Linked Web Storage server: finding the storage, why it needs the Solid Protocol's requests too, signing in with a token exchange, and what LWS can't give the app yet."@en . a schema:TechArticle ; diff --git a/docs/lws.md b/docs/lws.md index 9b0a12855..fb37c6d8c 100644 --- a/docs/lws.md +++ b/docs/lws.md @@ -23,8 +23,9 @@ doesn't ask a server to give: A server can speak both. The Community Solid Server with LWS (`css-lws`) does, and answers every request the app makes as a Solid -pod does. One that speaks LWS alone gets the app's requests made as LWS -has them, as below. +pod does. So does its LWS-alone configuration (`css-lws-only`). The app +doesn't work on a server that refuses the Solid Protocol's requests +([why](#why-the-app-needs-solid-requests)). ## Finding the storage @@ -53,58 +54,16 @@ answers with it, its Solid links kept. A server that lists a folder only as LWS has it answers too, where Turtle alone would get a 406. -## The bridge - -[lwsBridge.ts](../packages/solid/src/lwsBridge.ts) sits below the -fences (`podFence.ts`, `ownPodGate.ts`), on the user's fetch and on the -fetch as no one ([appUseCases.ts](../packages/composition/src/appUseCases.ts)). -It sends each request as the app makes it, and makes one as LWS has it -only when the server's answer asks for that: - -- **A folder's listing.** An answer in `application/lws+json` is read - as LWS gives it. A 406 to Turtle, or Turtle in the LWS profile, which - lists no `ldp:contains`, from a server that links its storage, gets - the listing asked for again in `application/lws+json`. The bridge - follows each page's `next` link on the same origin, and answers with - the Turtle listing the app reads, the first page's ETag and links - kept. -- **A new document or folder.** A PUT with `If-None-Match: *` answered - 404, 405, 415 or 501 by a server that links its storage is made - another way. The bridge makes each missing folder above it, then - POSTs the document to its folder, the name asked for in `Slug` as - the address spells it. A - resource the server names otherwise is deleted again, and the - creation is refused with 501. When the server made it where it - wouldn't say, or outside the folder, or the delete fails, the - creation is refused with 502, and nothing is deleted or tried again. -- **An edit.** A SPARQL Update PATCH (`INSERT DATA`, `DELETE DATA`) - answered the same way: the bridge reads the document, applies the - change, and PUTs the whole document with `If-Match`. A triple to - delete that isn't there fails the edit with 409, as a Solid server's - PATCH does. A PATCH that had no `If-Match`, the answer log's insert, - is written against the ETag read, never a weak one, and tried again - up to five times when another write came first, so every concurrent - answer is kept. A document not there yet is made as above. - -**Why it reacts to the server's answer.** A Solid server takes each -request as it's made, so it gets one request, as before. Which server -speaks which protocol is never configured: the server's answer says it. -Whether a server is an LWS server is asked by a HEAD of the resource or -the nearest folder above it that's there, whose answer must link its -storage. - -**Why the fences stay above it.** The pod fence and the credential gate -check each request as the app makes it, a write's target among them. -What the bridge sends besides goes to the same resource, the folders -above it, a resource its POST made in the folder, or the pages the -server links on its own origin. - -**Why a name the server changes is refused.** Every address in an -instance is the app's to choose ([data-model.md](data-model.md)): a -catalog names its decks' documents by address, and a type index names -the instance. A storage that names its resources itself can't hold an -instance until the app records where each resource went. LWS has an -open issue for a creation at a name the client chooses +## Why the app needs Solid requests + +The app makes each request as the Solid Protocol has it, and doesn't +translate it for a server that refuses them. Every server it's tested +on takes them. Translating them would help only part of the way: an +LWS server names a new resource itself, and the name a client asks for +is only a hint, but every address in an instance is the app's to choose +([data-model.md](data-model.md)). A catalog names its decks' documents +by address, and a type index names the instance. LWS has an open issue +for a creation at a name the client chooses ([w3c/lws-protocol#173](https://github.com/w3c/lws-protocol/issues/173)). ## Sign-in @@ -133,6 +92,8 @@ storage's. ## What LWS can't give the app yet +- **A server that takes LWS requests alone.** The app needs the Solid + Protocol's requests ([why](#why-the-app-needs-solid-requests)). - **A release anyone can read.** LWS defines no access control. The app writes a release's ACL or ACR where the server has one; the Community Solid Server with LWS keeps Web Access Control, so a release is made diff --git a/docs/standards.md b/docs/standards.md index 22b89bdae..1914d9609 100644 --- a/docs/standards.md +++ b/docs/standards.md @@ -20,7 +20,7 @@ Every specification below is open; the drafts and de facto ones are in | [Type Indexes](https://solid.github.io/type-indexes/) | 1.0.0 | Editor's draft | Finding and registering an instance: [`typeIndex.ts`](../packages/solid/src/typeIndex.ts) ([data-model.md](data-model.md#registrations)) | Yes (draft) | | [Web Access Control](https://solidproject.org/TR/wac) and [Access Control Policy](https://solidproject.org/TR/acp) | WAC 1.0.0; ACP 0.9.0 | W3C Solid Community Group drafts | Making a published release public: [`solidReleasePublisher.ts`](../packages/solid/src/solidReleasePublisher.ts), through `@inrupt/solid-client`'s universal access in [`client/access.ts`](../packages/solid/src/client/access.ts), which speaks whichever the pod uses ([boundaries.md](boundaries.md)) | Yes (draft) | | [Linked Data Platform](https://www.w3.org/TR/ldp/) | 1.0 | W3C Recommendation | Containers in the pod and the guest's pod: [`localPod.ts`](../packages/solid/src/localPod.ts) ([guest-mode.md](guest-mode.md)) | Yes | -| [Linked Web Storage Protocol](https://www.w3.org/TR/lws10-core/) and its [OpenID Connect authentication suite](https://www.w3.org/TR/lws10-authn-openid/) | 1.0 | W3C Working Draft | A storage that speaks LWS: finding it ([`lwsStorage.ts`](../packages/solid/src/lwsStorage.ts)), the bridge that makes the app's requests as LWS has them ([`lwsBridge.ts`](../packages/solid/src/lwsBridge.ts)), and its sign-in ([`lwsAuth.ts`](../packages/solid-memo-client/src/lwsAuth.ts)) ([lws.md](lws.md)) | Yes (draft) | +| [Linked Web Storage Protocol](https://www.w3.org/TR/lws10-core/) and its [OpenID Connect authentication suite](https://www.w3.org/TR/lws10-authn-openid/) | 1.0 | W3C Working Draft | A storage that speaks LWS: finding it ([`lwsStorage.ts`](../packages/solid/src/lwsStorage.ts)) and its sign-in ([`lwsAuth.ts`](../packages/solid-memo-client/src/lwsAuth.ts)), on a server that takes the Solid Protocol's requests too ([lws.md](lws.md)) | Yes (draft) | | [OAuth 2.0 Token Exchange](https://www.rfc-editor.org/rfc/rfc8693) and [Protected Resource Metadata](https://www.rfc-editor.org/rfc/rfc9728) | RFC 8693 (2020); RFC 9728 (2025) | IETF Proposed Standards | The ID token exchanged for an LWS storage's token, and the challenge that leads to it (`lwsAuth.ts`, [solid-memo-client.md](solid-memo-client.md#linked-web-storage)) | Yes | ## Linked data diff --git a/docs/testing.md b/docs/testing.md index ac323450e..6eb83fe4f 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -122,19 +122,6 @@ Community Solid Server release has LWS, or when its commit can't be fetched any more: a commit that can't be fetched fails the `contract` job of every pull request. -Every request of the tests goes through the bridge to an LWS server -(`lwsBridge.ts`, `setup.ts`), as the app's do, -and the contract's too for a server `SOLID_SERVER_URL` names. A Solid -server takes each request as it's made, so the bridge sends it on as -it is. The fork takes the Solid Protocol's requests even with LWS -alone, so the bridge's own way of making them shows only on a server -that refuses them. `npm run lws:strict -w @solid-memo/e2e-pod -- - ` runs one ([strictLws.ts](../e2e/pod/strictLws.ts)): a -proxy that refuses a folder's listing in Turtle, a PUT of a resource -not there, and every PATCH. With `E2E_SEEDS_AS_LWS=1`, a test's own -seed, a plain PUT of a document not there, is made as LWS makes one. CI -runs neither. - Each server is a compose file, `e2e/pod/servers//compose.yml`, asking only for `E2E_PORT` (the port on 127.0.0.1 the harness picked, which the server must be told) and `E2E_SECRET` (a password made up for the run). It publishes on @@ -392,7 +379,7 @@ The tests check that: On `css-lws` and `css-lws-only`, the sign-in client is the app's own ([signedInClient.ts](../e2e/pod/signedInClient.ts)): `solid-memo-client`'s -session, below the bridge, as the app wires it, since Inrupt's client +session, as the app wires it, since Inrupt's client gives the app no ID token. So the requests are the ones a browser's session makes, and: diff --git a/e2e/pod/contract.ts b/e2e/pod/contract.ts index 4e3f137e1..f14be35c8 100644 --- a/e2e/pod/contract.ts +++ b/e2e/pod/contract.ts @@ -13,18 +13,12 @@ const LDP_CONTAINS = "http://www.w3.org/ns/ldp#contains"; * lists it at the URL it was written to (so the server knows the URL it * is reached at: a wrong base URL, port or trailing slash shows here, * not as every test failing), and it deletes. Throws saying which failed. - * The requests go through `fetch`: the bridge to a Linked Web Storage - * server (lwsBridge.ts) for a server a contributor names, which may be one. */ -export async function checkContract( - root: string, - fetch: typeof globalThis.fetch = globalThis.fetch, -): Promise { +export async function checkContract(root: string): Promise { const url = new URL(`probe-${randomUUID()}.ttl`, root).href; const put = await fetch(url, { method: "PUT", - // Made as the app makes a document: only where none is. - headers: { "content-type": "text/turtle", "if-none-match": "*" }, + headers: { "content-type": "text/turtle" }, body: `<#a> <#b> "1" .`, }).catch((error: Error) => { throw new Error( diff --git a/e2e/pod/globalSetup.ts b/e2e/pod/globalSetup.ts index fc7a8c314..15dab15e4 100644 --- a/e2e/pod/globalSetup.ts +++ b/e2e/pod/globalSetup.ts @@ -3,7 +3,6 @@ // SPDX-License-Identifier: MIT import type { TestProject } from "vitest/node"; -import { lwsBridge } from "@solid-memo/solid/lwsBridge"; import { checkContract } from "./contract"; import { serversNamed, @@ -37,8 +36,7 @@ export default async function setup(project: TestProject) { const given = process.env.SOLID_SERVER_URL; if (given !== undefined && given !== "") { const url = given.endsWith("/") ? given : `${given}/`; - // Made as the app makes its requests, through the bridge to a Linked Web Storage server it may be (setup.ts). - await checkContract(url, lwsBridge(fetch)); + await checkContract(url); project.provide("solidServers", [ { id: "external", tier: "advisory", name: process.env.SOLID_SERVER_NAME || given, url }, ]); diff --git a/e2e/pod/package.json b/e2e/pod/package.json index 760404d9c..18226b93c 100644 --- a/e2e/pod/package.json +++ b/e2e/pod/package.json @@ -13,8 +13,7 @@ "test:pod": "vitest run", "pod": "node pod.ts", "servers": "node servers.ts prepare", - "pod:clean": "node servers.ts clean", - "lws:strict": "node strictLws.ts" + "pod:clean": "node servers.ts clean" }, "dependencies": { "@solid-memo/application": "*", diff --git a/e2e/pod/setup.ts b/e2e/pod/setup.ts index 06e8afbc8..d4acd1266 100644 --- a/e2e/pod/setup.ts +++ b/e2e/pod/setup.ts @@ -2,7 +2,6 @@ // // SPDX-License-Identifier: MIT -import { lwsBridge } from "@solid-memo/solid/lwsBridge"; import { closingConnections } from "./connections.ts"; const send = closingConnections(globalThis.fetch); @@ -13,37 +12,7 @@ const send = closingConnections(globalThis.fetch); * module keeps the fetch it was given. None follows a redirect, as none * the app sends with the user's credentials does (authFetch.ts), so a * server that answers a pod request with one fails here. A scripted login - * reads each redirect it gets, by asking for "manual" (signedIn.ts). A request a - * Linked Web Storage server does not take as the app makes it is made as - * LWS has it, as the app's are (lwsBridge.ts, docs/lws.md); to any other - * server it is sent as it is. + * reads each redirect it gets, by asking for "manual" (signedIn.ts). */ -const bridged = lwsBridge((input, init) => - send(input, { ...init, redirect: init?.redirect === "manual" ? "manual" : "error" }), -); - -/** - * With E2E_SEEDS_AS_LWS=1, for a run on a server that takes only LWS - * (strictLws.ts): what a test writes with a plain PUT, as another app - * writes its data, and the server answers 404, not there yet, is made - * as LWS makes a resource, as the app makes a document (with - * `If-None-Match: *`, which every creation of the app's sends). The - * app's own requests are never so made again: none is a plain PUT of - * something not there (docs/lws.md). - */ -const seedsAsLws = process.env.E2E_SEEDS_AS_LWS === "1"; - -globalThis.fetch = async (input, init) => { - const response = await bridged(input, init); - const headers = new Headers(init?.headers); - if ( - !seedsAsLws || - init?.method !== "PUT" || - headers.has("If-None-Match") || - headers.has("If-Match") || - response.status !== 404 - ) - return response; - headers.set("If-None-Match", "*"); - return bridged(input, { ...init, headers }); -}; +globalThis.fetch = (input, init) => + send(input, { ...init, redirect: init?.redirect === "manual" ? "manual" : "error" }); diff --git a/e2e/pod/signedInClient.ts b/e2e/pod/signedInClient.ts index f82e52659..beffcaa82 100644 --- a/e2e/pod/signedInClient.ts +++ b/e2e/pod/signedInClient.ts @@ -4,7 +4,6 @@ import { createPublicKey } from "node:crypto"; import { OidcSession, setCurrentSession } from "@solid-memo/solid-memo-client/oidcSession"; -import { lwsBridge } from "@solid-memo/solid/lwsBridge"; import type { SessionClient } from "@solid-memo/solid/client/session"; import { sessionClient, type Session } from "./signedIn.ts"; @@ -13,8 +12,7 @@ const ECDSA = { name: "ECDSA", namedCurve: "P-256" }; /** * The sign-in client over a session. Where it has an ID token, that of * the app's own Solid client, whose session makes a Linked Web Storage - * server's token exchange (docs/lws.md), with the app's bridge above it, - * as appUseCases.ts wires it. It is apart from signedIn.ts because + * server's token exchange (docs/lws.md). It is apart from signedIn.ts because * servers.ts, which node runs as it is, imports that. */ export function clientOf(session: Session): SessionClient { @@ -43,10 +41,10 @@ export function clientOf(session: Session): SessionClient { ); return { ...plain, - fetch: lwsBridge((input, init) => { + fetch: (input, init) => { // The app has one user at a time; these tests have two, one request at a time. setCurrentSession(oidc); return oidc.fetch(input, init); - }), + }, }; } diff --git a/e2e/pod/strictLws.test.ts b/e2e/pod/strictLws.test.ts deleted file mode 100644 index 220d9be15..000000000 --- a/e2e/pod/strictLws.test.ts +++ /dev/null @@ -1,53 +0,0 @@ -// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors -// -// SPDX-License-Identifier: MIT - -import { describe, expect, it } from "vitest"; -import { ruleFor, targetOf } from "./strictLws.ts"; - -describe("the strict LWS proxy's rules", () => { - it("lists a container only as LWS does, and refuses Turtle alone", () => { - expect(ruleFor("GET", "/a/", "text/turtle")).toEqual({ refuse: 406 }); - expect(ruleFor("HEAD", "/a/", "text/turtle, */*;q=0.1")).toEqual({ - send: true, - accept: "application/lws+json", - }); - expect(ruleFor("GET", "/a/", "application/lws+json")).toEqual({ - send: true, - accept: "application/lws+json", - }); - expect(ruleFor("GET", "/a/", undefined)).toEqual({ - send: true, - accept: "application/lws+json", - }); - expect(ruleFor("GET", "/", "application/lws+cid")).toEqual({ send: true }); - }); - - it("refuses every PATCH and a PUT of a resource not there, and sends the rest on", () => { - expect(ruleFor("PATCH", "/a/b.ttl", undefined)).toEqual({ refuse: 415 }); - expect(ruleFor("PUT", "/a/b.ttl", undefined)).toEqual({ sendIfThere: true }); - expect(ruleFor("GET", "/a/b.ttl", "text/turtle")).toEqual({ send: true }); - expect(ruleFor("POST", "/a/", undefined)).toEqual({ send: true }); - expect(ruleFor("DELETE", "/a/b.ttl", undefined)).toEqual({ send: true }); - }); -}); - -describe("the strict LWS proxy's target", () => { - const server = "http://127.0.0.1:3000/"; - - it("keeps a request's path and query", () => { - expect(targetOf(server, "/a/b.ttl?c=d").href).toBe("http://127.0.0.1:3000/a/b.ttl?c=d"); - expect(targetOf(server, "/a/../b/").href).toBe("http://127.0.0.1:3000/b/"); - }); - - it("stays on the server, whatever host a request line names", () => { - for (const requestTarget of [ - "http://elsewhere.example/a", - "//elsewhere.example/a", - "/\\elsewhere.example/a", - "/.//elsewhere.example/a", - "http://user:password@elsewhere.example:8080/a", - ]) - expect(targetOf(server, requestTarget).host).toBe("127.0.0.1:3000"); - }); -}); diff --git a/e2e/pod/strictLws.ts b/e2e/pod/strictLws.ts deleted file mode 100644 index fa54fa146..000000000 --- a/e2e/pod/strictLws.ts +++ /dev/null @@ -1,128 +0,0 @@ -// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors -// -// SPDX-License-Identifier: MIT - -/** - * `npm run lws:strict -w @solid-memo/e2e-pod -- `: a proxy - * on 127.0.0.1: in front of a Linked Web Storage server that also - * takes the Solid Protocol's requests (the Community Solid Server with - * LWS, css-lws-only), which refuses what LWS Protocol 1.0 does not ask a - * server to take, so that the end-to-end tests show what the app does - * on a server that takes only LWS (docs/lws.md): a container listed in - * Turtle (406; an LWS server lists one as `application/lws+json`), a - * PUT of a resource not there (404; LWS makes one with a POST to its - * container), and any PATCH (415; LWS requires only JSON Patch, which - * the app does not send). The server must be told the proxy's URL as - * its base URL. For a contributor's own runs; CI runs none. - */ -import { createServer, type IncomingMessage, type ServerResponse } from "node:http"; - -/** What the proxy does with a request: send it on (with this Accept, if given), ask first whether it is there, or refuse it. */ -export type Rule = - | { send: true; accept?: string } - | { sendIfThere: true } - | { refuse: 404 | 406 | 415 }; - -const LWS_JSON = "application/lws+json"; - -/** The rule for a request: its method, its path, and its Accept header. */ -export function ruleFor(method: string, path: string, accept: string | undefined): Rule { - if ((method === "GET" || method === "HEAD") && path.endsWith("/")) { - const asked = accept ?? "*/*"; - // A storage's description, at its URI, which is its root container's too, is sent on as asked for. - if (asked.includes("application/lws+cid")) return { send: true }; - if (asked.includes(LWS_JSON) || asked.includes("*/*")) return { send: true, accept: LWS_JSON }; - return { refuse: 406 }; - } - if (method === "PATCH") return { refuse: 415 }; - if (method === "PUT") return { sendIfThere: true }; - return { send: true }; -} - -/** - * The URL on the server that a request's target asks for: only its path - * and query, for a request line may name a host of its own - * (`GET http://elsewhere/ HTTP/1.1`, `GET //elsewhere/`), which the proxy - * does not go to. - */ -export function targetOf(server: string, requestTarget: string): URL { - const { pathname, search } = new URL(requestTarget, server); - const target = new URL(server); - target.pathname = pathname; - target.search = search; - return target; -} - -/** The headers of a response a proxy may pass on as they are (fetch has undone its encoding and length). */ -const DROPPED = new Set([ - "content-encoding", - "content-length", - "transfer-encoding", - "connection", - "keep-alive", -]); - -async function bodyOf(request: IncomingMessage): Promise { - const chunks: Uint8Array[] = []; - for await (const chunk of request) chunks.push(new Uint8Array(chunk as Buffer)); - return chunks.length === 0 ? undefined : new Blob(chunks); -} - -function headersOf(request: IncomingMessage, accept?: string): Headers { - const headers = new Headers(); - for (const [name, value] of Object.entries(request.headers)) - if (value !== undefined && name !== "host" && name !== "connection") - headers.set(name, Array.isArray(value) ? value.join(", ") : value); - // The server names its resources by the host it is asked at, the proxy's (its base URL). - headers.set("x-forwarded-host", request.headers.host ?? ""); - headers.set("x-forwarded-proto", "http"); - if (accept !== undefined) headers.set("accept", accept); - return headers; -} - -async function proxy(server: string, request: IncomingMessage, response: ServerResponse) { - const method = request.method ?? "GET"; - const target = targetOf(server, request.url ?? "/"); - const rule = ruleFor(method, target.pathname, request.headers.accept); - const body = await bodyOf(request); - if ("refuse" in rule) { - response.writeHead(rule.refuse, { "access-control-allow-origin": "*" }).end(); - return; - } - if ("sendIfThere" in rule) { - const there = await fetch(target, { method: "HEAD", headers: headersOf(request) }); - if (there.status === 404) { - response.writeHead(404).end(); - return; - } - } - const answer = await fetch(target, { - method, - headers: headersOf(request, "accept" in rule ? rule.accept : undefined), - body, - redirect: "manual", - }); - const headers: Record = {}; - answer.headers.forEach((value, name) => { - if (!DROPPED.has(name)) headers[name] = [...(headers[name] ?? []), value]; - }); - response.writeHead(answer.status, headers); - response.end(method === "HEAD" ? undefined : Buffer.from(await answer.arrayBuffer())); -} - -if (import.meta.main) { - const [server, port] = process.argv.slice(2); - if (server === undefined || port === undefined) { - console.error("Usage: node strictLws.ts "); - process.exit(2); - } - createServer((request, response) => { - proxy(server, request, response).catch((error: Error) => { - response.writeHead(502).end(error.message); - }); - }).listen(Number(port), "127.0.0.1", () => - console.log( - `A server that takes only LWS, in front of ${server}, at http://127.0.0.1:${port}/`, - ), - ); -} diff --git a/packages/composition/src/appUseCases.ts b/packages/composition/src/appUseCases.ts index 444ed8d4d..2e63620c8 100644 --- a/packages/composition/src/appUseCases.ts +++ b/packages/composition/src/appUseCases.ts @@ -6,7 +6,6 @@ import { createUseCases, type UseCases } from "@solid-memo/application/useCases" import { createPodFence } from "@solid-memo/solid/podFence"; import { createWriteFence } from "@solid-memo/solid/writeFence"; import { anonymousFetch } from "@solid-memo/solid/authFetch"; -import { lwsBridge } from "@solid-memo/solid/lwsBridge"; import { withReadDeadline } from "@solid-memo/solid/readDeadline"; import { createSolidSession } from "@solid-memo/solid/solidSession"; import { createShaclShapeValidator } from "@solid-memo/solid/shaclShapeValidator"; @@ -110,19 +109,17 @@ export function createAppUseCases({ * (docs/guest-mode.md); then to the guest's pod or, as the logged-in * user, to any other, following no redirect to an address the fences * never checked (authFetch.ts). A read of another server that gets no - * answer in time is given up, to be tried again (readDeadline.ts). A - * request a Linked Web Storage server does not take as the app makes - * it is made as LWS has it (lwsBridge.ts, docs/lws.md). + * answer in time is given up, to be tried again (readDeadline.ts). */ const writeFence = createWriteFence( routedFetch({ origin: GUEST_ORIGIN, local: guestFetch, - remote: withReadDeadline(lwsBridge(session.fetch)), + remote: withReadDeadline(session.fetch), }), ); /** A fetch as no one (authFetch.ts), whose reads have a deadline as the user's do. */ - const noOneFetch = withReadDeadline(lwsBridge(anonymousFetch)); + const noOneFetch = withReadDeadline(anonymousFetch); const podFetch = createPodFence(writeFence.fetch); /** The release publisher's alone, for a release's own access control: the pod fence lets it write a document's ACL or ACR. */ const shareFetch = createPodFence(writeFence.fetch, { accessControl: true }); diff --git a/packages/solid/src/lwsBridge.test.ts b/packages/solid/src/lwsBridge.test.ts deleted file mode 100644 index afb0046dd..000000000 --- a/packages/solid/src/lwsBridge.test.ts +++ /dev/null @@ -1,468 +0,0 @@ -// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors -// -// SPDX-License-Identifier: MIT - -import { describe, expect, it } from "vitest"; -import { listContainer } from "./containers"; -import { appendToDocument, deleteDataset, readDataset, saveDataset } from "./datasets"; -import { buildThing, createSolidDataset, createThing, getThing, setThing } from "./client/rdf"; -import { CREATION_REFUSED, lwsBridge } from "./lwsBridge"; -import { lwsPod } from "./testing/lwsPod"; - -const ROOT = "https://pod.example/alice/"; -const MAIN = `${ROOT}solid-memo/main/`; -const CATALOG = `${MAIN}catalog.ttl`; -const MONTH = `${MAIN}history/2026-11.ttl`; -const TITLE = "http://purl.org/dc/terms/title"; -const STORAGE_LINK = `<${ROOT}>; rel="https://www.w3.org/ns/lws#storage"`; - -function titled(url: string, title: string) { - return buildThing(createThing({ url })).addStringNoLocale(TITLE, title).build(); -} - -/** A fetch whose answers `answer` gives, each with the URL asked for; every request kept in `asked`. */ -function scripted( - answer: (url: string, init: RequestInit | undefined, n: number) => Response | Promise, -) { - const asked: string[] = []; - const fetch: typeof globalThis.fetch = async (input, init) => { - asked.push(`${init?.method ?? "GET"} ${String(input)}`); - const response = await answer(String(input), init, asked.length); - if (response.url === "") Object.defineProperty(response, "url", { value: String(input) }); - return response; - }; - return { fetch, asked }; -} - -const lws = (status: number, headers: Record = {}, body: string | null = null) => - new Response(body, { status, headers: { Link: STORAGE_LINK, ...headers } }); - -describe("the app's documents on a server that takes only Linked Web Storage", () => { - it("makes each folder and document with a POST of its name, edits them whole, lists and deletes them", async () => { - const pod = lwsPod({ root: ROOT }); - const fetch = lwsBridge(pod.fetch); - await saveDataset( - CATALOG, - setThing(createSolidDataset(), titled(`${CATALOG}#it`, "one")), - fetch, - ); - expect(pod.resources.get(CATALOG)?.body).toContain('"one"'); - expect(pod.requests.filter((request) => request.startsWith("POST"))).toEqual([ - `POST ${ROOT}`, - `POST ${ROOT}solid-memo/`, - `POST ${MAIN}`, - ]); - - const read = await readDataset(CATALOG, fetch); - const changed = buildThing(getThing(read, `${CATALOG}#it`)!) - .setStringNoLocale(TITLE, "two") - .build(); - await saveDataset(CATALOG, setThing(read, changed), fetch); - expect(pod.resources.get(CATALOG)?.body).toContain('"two"'); - expect(pod.resources.get(CATALOG)?.body).not.toContain('"one"'); - expect(pod.requests).toContain(`PATCH ${CATALOG}`); - expect(pod.requests.at(-1)).toBe(`PUT ${CATALOG}`); - - expect(await listContainer(MAIN, fetch)).toEqual([CATALOG]); - await deleteDataset(CATALOG, await readDataset(CATALOG, fetch), fetch); - expect(pod.resources.has(CATALOG)).toBe(false); - expect(await listContainer(MAIN, fetch)).toEqual([]); - }); - - it("appends to a document not there yet, and keeps every one of several appends made at once", async () => { - const pod = lwsPod({ root: ROOT }); - const fetch = lwsBridge(pod.fetch); - await appendToDocument(MONTH, titled(`${MONTH}#a`, "a"), fetch); - await Promise.all( - ["b", "c", "d"].map((name) => - appendToDocument(MONTH, titled(`${MONTH}#${name}`, name), fetch), - ), - ); - for (const name of ["a", "b", "c", "d"]) - expect(pod.resources.get(MONTH)?.body).toContain(`"${name}"`); - expect(await listContainer(`${MAIN}history/`, fetch)).toEqual([MONTH]); - }); - - it("reads a listing in pages, and one a server gives only in the LWS profile", async () => { - for (const options of [{ pageSize: 2 }, { profileTurtle: true }]) { - const pod = lwsPod({ root: ROOT, ...options }); - const fetch = lwsBridge(pod.fetch); - const documents = ["a", "b", "c", "d", "e"].map((name) => `${MAIN}${name}.ttl`); - for (const document of documents) - await saveDataset( - document, - setThing(createSolidDataset(), titled(`${document}#it`, "x")), - fetch, - ); - expect(await listContainer(MAIN, fetch)).toEqual(documents); - } - }); - - it("refuses a document the server would name otherwise, and leaves nothing of it", async () => { - const pod = lwsPod({ root: ROOT, ownNames: true }); - const fetch = lwsBridge(pod.fetch); - await expect( - saveDataset(CATALOG, setThing(createSolidDataset(), titled(`${CATALOG}#it`, "one")), fetch), - ).rejects.toMatchObject({ response: { status: CREATION_REFUSED } }); - expect([...pod.resources.keys()]).toEqual([ROOT]); - }); -}); - -describe("lwsBridge, on a Solid server", () => { - it("sends every request as it is when the server takes it, or is no LWS server", async () => { - const { fetch, asked } = scripted((_, init) => { - if (init?.method === "HEAD") return new Response(null); - if (init?.method === "PUT" || init?.method === "PATCH") - return new Response(null, { status: 404 }); - return new Response("", { headers: { "Content-Type": "text/turtle" } }); - }); - const bridged = lwsBridge(fetch); - expect((await bridged(MAIN)).status).toBe(200); - expect( - (await bridged(CATALOG, { method: "PUT", headers: { "If-None-Match": "*" }, body: "" })) - .status, - ).toBe(404); - expect( - ( - await bridged(CATALOG, { - method: "PATCH", - headers: { "Content-Type": "application/sparql-update" }, - body: "", - }) - ).status, - ).toBe(404); - expect((await bridged(CATALOG, { method: "DELETE" })).status).toBe(200); - expect((await bridged(CATALOG, { method: "PATCH", body: "" })).status).toBe(404); - const request = new Request(CATALOG); - expect((await bridged(request)).status).toBe(200); - expect(asked).toEqual([ - `GET ${MAIN}`, - `PUT ${CATALOG}`, - `HEAD ${MAIN}`, - `PATCH ${CATALOG}`, - `HEAD ${CATALOG}`, - `DELETE ${CATALOG}`, - `PATCH ${CATALOG}`, - "GET [object Request]", - ]); - }); - - it("asks nothing more of a server that takes a creation or an update, or refuses it for another reason", async () => { - for (const status of [201, 412, 403, 400]) { - const { fetch, asked } = scripted(() => new Response(null, { status })); - const bridged = lwsBridge(fetch); - await bridged(CATALOG, { method: "PUT", headers: { "If-None-Match": "*" }, body: "" }); - await bridged(CATALOG, { - method: "PATCH", - headers: { "Content-Type": "application/sparql-update" }, - body: "", - }); - expect(asked).toHaveLength(2); - } - }); - - it("is no LWS server when no folder up to the root is there, or there are too many", async () => { - const { fetch, asked } = scripted(() => new Response(null, { status: 404 })); - await lwsBridge(fetch)("https://pod.example/a/b.ttl", { - method: "PUT", - headers: { "If-None-Match": "*" }, - }); - expect(asked).toEqual([ - "PUT https://pod.example/a/b.ttl", - "HEAD https://pod.example/a/", - "HEAD https://pod.example/", - ]); - const deep = `https://pod.example/${"x/".repeat(40)}d.ttl`; - const many = scripted(() => new Response(null, { status: 404 })); - await lwsBridge(many.fetch)(deep, { method: "PUT", headers: { "If-None-Match": "*" } }); - expect(many.asked).toHaveLength(33); - // The origin's root itself has no container to be made in. - const root = scripted(() => new Response(null, { status: 404 })); - await lwsBridge(root.fetch)("https://pod.example/", { - method: "PUT", - headers: { "If-None-Match": "*" }, - }); - expect(root.asked).toHaveLength(1); - }); -}); - -describe("lwsBridge, its listing", () => { - const listing = (body: unknown, headers: Record = {}) => - lws(200, { "Content-Type": "application/lws+json", ...headers }, JSON.stringify(body)); - - it("keeps the members of the container alone, and fails a listing that is not one", async () => { - const { fetch } = scripted(() => - listing({ - items: [{ id: `${MAIN}a.ttl` }, { id: 7 }, null, { id: `${ROOT}other.ttl` }, { id: MAIN }], - }), - ); - expect(await (await lwsBridge(fetch)(MAIN)).text()).toBe( - `<${MAIN}> a , ;\n <${MAIN}a.ttl> .\n`, - ); - const broken = scripted(() => listing({ type: "Container" })); - await expect(lwsBridge(broken.fetch)(MAIN)).rejects.toThrow( - "is not the listing of a container", - ); - }); - - it("follows a listing's pages on its origin alone, each once, and no further than it reads", async () => { - const next = (url: string) => ({ Link: `<${url}>; rel="next"` }); - for (const [link, message] of [ - ["https://elsewhere.example/p2", "goes on past"], - [MAIN, "goes on past"], - ] as const) { - const { fetch } = scripted(() => listing({ items: [] }, next(link))); - await expect(lwsBridge(fetch)(MAIN)).rejects.toThrow(message); - } - const endless = scripted((_, __, n) => listing({ items: [] }, next(`${MAIN}?p=${n}`))); - await expect(lwsBridge(endless.fetch)(MAIN)).rejects.toThrow("goes on past"); - expect(endless.asked).toHaveLength(200); - const failing = scripted((_, __, n) => - n === 1 ? listing({ items: [] }, next(`${MAIN}?p=2`)) : lws(500), - ); - await expect(lwsBridge(failing.fetch)(MAIN)).rejects.toThrow("answered 500"); - }); - - it("refuses a listing of more members than it reads", async () => { - const items = Array.from({ length: 20_001 }, (_, n) => ({ id: `${MAIN}${n}.ttl` })); - const { fetch } = scripted(() => listing({ items })); - await expect(lwsBridge(fetch)(MAIN)).rejects.toThrow("lists more than 20000"); - }); - - it("asks for the server's own listing only when the answer names the LWS profile as its profile", async () => { - const PROFILE = "https://www.w3.org/ns/lws/v1"; - const requestsFor = async (contentType: string | undefined) => { - const headers: Record = contentType ? { "Content-Type": contentType } : {}; - const { fetch, asked } = scripted(() => lws(200, headers)); - await lwsBridge(fetch)(MAIN); - return asked.length; - }; - for (const named of [ - `text/turtle; profile="${PROFILE}"`, - `text/turtle;PROFILE = "https://other.example/p ${PROFILE}"`, - `text/turtle; charset=utf-8; profile=${PROFILE}`, - ]) - expect(await requestsFor(named)).toBe(3); - for (const unnamed of [ - undefined, - "text/turtle", - `text/turtle; profile="${PROFILE}.elsewhere.example"`, - `text/turtle; profile=https://elsewhere.example/${PROFILE}`, - `text/turtle; note="${PROFILE}"`, - `text/turtle; profile=`, - ]) - expect(await requestsFor(unnamed)).toBe(1); - }); - - it("names the listing by the address asked for when the answer has none, and passes on one of no type", async () => { - const bare: typeof globalThis.fetch = async () => - new Response(JSON.stringify({ items: [{ id: "a.ttl" }] }), { - headers: { "Content-Type": "application/lws+json" }, - }); - const response = await lwsBridge(bare)(MAIN); - expect(response.url).toBe(MAIN); - expect(await response.text()).toContain(`<${MAIN}a.ttl>`); - const untyped: typeof globalThis.fetch = async () => new Response("x"); - expect(await (await lwsBridge(untyped)(MAIN)).text()).toBe("x"); - }); - - it("asks for its listing a server that will not give Turtle, and gives its answer when it has none", async () => { - const { fetch, asked } = scripted((_, init, n) => - n === 1 ? lws(406) : init?.method === "HEAD" ? lws(200) : lws(404), - ); - expect((await lwsBridge(fetch)(MAIN, { headers: { Accept: "text/turtle" } })).status).toBe(404); - expect(asked).toEqual([`GET ${MAIN}`, `HEAD ${MAIN}`, `GET ${MAIN}`]); - const plain = scripted((_, __, n) => - n === 1 ? lws(406) : lws(200, { "Content-Type": "text/plain" }), - ); - expect((await lwsBridge(plain.fetch)(MAIN)).headers.get("Content-Type")).toBe("text/plain"); - }); - - it("asks a server that is no LWS server for nothing more, whatever it answers a listing", async () => { - const { fetch, asked } = scripted((_, init) => - init?.method === "HEAD" ? new Response(null) : new Response(null, { status: 406 }), - ); - expect((await lwsBridge(fetch)(MAIN)).status).toBe(406); - expect(asked).toEqual([`GET ${MAIN}`, `HEAD ${MAIN}`]); - }); - - it("writes a member whose address holds what an IRI may not as an IRI", async () => { - const { fetch } = scripted(() => - lws( - 200, - { "Content-Type": "application/lws+json" }, - JSON.stringify({ items: [{ id: `${MAIN}a|b^c.ttl` }] }), - ), - ); - expect(await (await lwsBridge(fetch)(MAIN)).text()).toContain(`<${MAIN}a%7Cb%5Ec.ttl>`); - }); -}); - -describe("lwsBridge, its creation", () => { - const creating = { - method: "PUT", - headers: { "If-None-Match": "*", "Content-Type": "text/turtle" }, - body: "<#a> <#b> <#c>.", - }; - - it("says why a folder above could not be made, and what the server answered a POST", async () => { - const forbidden = scripted((url, init) => - init?.method === "HEAD" && url === MAIN ? lws(403) : lws(404), - ); - expect((await lwsBridge(forbidden.fetch)(CATALOG, creating)).status).toBe(403); - const refused = scripted((url, init) => - init?.method === "POST" - ? lws(400) - : init?.method === "HEAD" && url === MAIN - ? lws(200) - : lws(404), - ); - expect((await lwsBridge(refused.fetch)(CATALOG, creating)).status).toBe(400); - const deep = `${ROOT}${"x/".repeat(40)}d.ttl`; - const tooDeep = scripted((url, init) => - init?.method === "HEAD" && url === ROOT ? lws(200) : lws(404), - ); - expect((await lwsBridge(tooDeep.fetch)(deep, creating)).status).toBe(404); - }); - - it("deletes nothing but what the POST made in the folder, and never tries again what it could not delete", async () => { - for (const [location, deleteStatus] of [ - [null, 204], - ["https://elsewhere.example/alice/x.ttl", 204], - [`${ROOT}x.ttl`, 204], - [`${MAIN}x.ttl`, 500], - ] as const) { - const { fetch, asked } = scripted((url, init) => { - const method = init?.method ?? "GET"; - if (method === "POST") return lws(201, location === null ? {} : { Location: location }); - if (method === "HEAD") return url === MAIN ? lws(200) : lws(404); - if (method === "DELETE") return lws(deleteStatus); - return lws(405); - }); - const response = await lwsBridge(fetch)(CATALOG, creating); - expect(response.status).toBe(502); - expect(response.url).toBe(CATALOG); - const deletes = asked.filter((request) => request.startsWith("DELETE")); - expect(deletes).toEqual(location === `${MAIN}x.ttl` ? [`DELETE ${MAIN}x.ttl`] : []); - } - }); - - it("asks for a name outside ASCII as the address spells it", async () => { - const pod = lwsPod({ root: ROOT }); - const document = `${ROOT}한국어/카드.ttl`; - await saveDataset( - document, - setThing(createSolidDataset(), titled(`${document}#it`, "x")), - lwsBridge(pod.fetch), - ); - expect(pod.resources.has(new URL(document).href)).toBe(true); - }); - - it("makes a folder asked for with a POST that links its type, with no body", async () => { - const pod = lwsPod({ root: ROOT }); - const response = await lwsBridge(pod.fetch)(`${MAIN}decks/`, { - method: "PUT", - headers: { - "If-None-Match": "*", - "Content-Type": "text/turtle", - Link: '; rel="type"', - }, - }); - expect(response.status).toBe(201); - expect(response.url).toBe(`${MAIN}decks/`); - expect(pod.resources.get(`${MAIN}decks/`)).toMatchObject({ body: "" }); - }); -}); - -describe("lwsBridge, its update", () => { - const PATCH = (body: string, headers: Record = {}) => ({ - method: "PATCH", - headers: { "Content-Type": "application/sparql-update", ...headers }, - body, - }); - const INSERT = `INSERT DATA { <${CATALOG}#a> <${TITLE}> "a" . };`; - - it("does not write an update the app does not make, or one of a document with no version", async () => { - const pod = lwsPod({ root: ROOT }); - const fetch = lwsBridge(pod.fetch); - expect((await fetch(CATALOG, PATCH("DELETE WHERE { ?s ?p ?o }"))).status).toBe(415); - await saveDataset( - CATALOG, - setThing(createSolidDataset(), titled(`${CATALOG}#it`, "one")), - fetch, - ); - expect((await fetch(CATALOG, PATCH("DELETE WHERE { ?s ?p ?o }"))).status).toBe(415); - const unversioned = scripted((_, init) => - (init?.method ?? "GET") === "GET" - ? lws(200, { "Content-Type": "text/turtle" }, "") - : lws(415), - ); - expect((await lwsBridge(unversioned.fetch)(CATALOG, PATCH(INSERT))).status).toBe(415); - }); - - it("says another write came first to an edit of a document as read that is gone, or changed", async () => { - const pod = lwsPod({ root: ROOT }); - const fetch = lwsBridge(pod.fetch); - expect((await fetch(CATALOG, PATCH(INSERT, { "If-Match": '"1"' }))).status).toBe(412); - await fetch(CATALOG, PATCH(INSERT)); - expect((await fetch(CATALOG, PATCH(INSERT, { "If-Match": '"0"' }))).status).toBe(412); - }); - - it("gives up after a few tries, and passes on a read that fails", async () => { - const busy = scripted((_, init) => - (init?.method ?? "GET") === "GET" - ? lws(200, { "Content-Type": "text/turtle", ETag: '"1"' }, "") - : init?.method === "PUT" - ? lws(412) - : lws(415), - ); - expect((await lwsBridge(busy.fetch)(CATALOG, PATCH(INSERT))).status).toBe(412); - expect(busy.asked.filter((request) => request.startsWith("PUT"))).toHaveLength(5); - const failing = scripted((_, init) => - (init?.method ?? "GET") === "GET" ? lws(403) : lws(415), - ); - expect((await lwsBridge(failing.fetch)(CATALOG, PATCH(INSERT))).status).toBe(403); - }); - - it("writes nothing against a weak ETag, and fails an update whose deletion is not there", async () => { - const weak = scripted((_, init) => - (init?.method ?? "GET") === "GET" - ? lws(200, { "Content-Type": "text/turtle", ETag: 'W/"1"' }, "") - : lws(415), - ); - expect((await lwsBridge(weak.fetch)(CATALOG, PATCH(INSERT))).status).toBe(415); - expect(weak.asked.some((request) => request.startsWith("PUT"))).toBe(false); - const pod = lwsPod({ root: ROOT }); - const fetch = lwsBridge(pod.fetch); - const DELETE = `DELETE DATA { <${CATALOG}#b> <${TITLE}> "b" . }; ${INSERT}`; - expect((await fetch(CATALOG, PATCH(DELETE))).status).toBe(409); - expect(pod.resources.has(CATALOG)).toBe(false); - await fetch(CATALOG, PATCH(INSERT)); - expect((await fetch(CATALOG, PATCH(DELETE))).status).toBe(409); - }); - - it("says another write came first, not that a deletion isn't there, to an edit of a document changed since it was read", async () => { - const pod = lwsPod({ root: ROOT }); - const fetch = lwsBridge(pod.fetch); - await fetch(CATALOG, PATCH(INSERT)); - const DELETE = `DELETE DATA { <${CATALOG}#b> <${TITLE}> "b" . }; ${INSERT}`; - expect((await fetch(CATALOG, PATCH(DELETE, { "If-Match": '"0"' }))).status).toBe(412); - }); - - it("changes the document another tab made first by the same name", async () => { - let made = false; - const { fetch } = scripted((url, init) => { - const method = init?.method ?? "GET"; - if (method === "PATCH") return lws(404); - if (method === "HEAD") return url === MAIN || url === `${MAIN}history/` ? lws(200) : lws(404); - if (method === "GET") - return made ? lws(200, { "Content-Type": "text/turtle", ETag: '"1"' }, "") : lws(404); - if (method === "POST") { - made = true; - return lws(201, { Location: `${MAIN}history/2026-11-2.ttl` }); - } - return lws(204); - }); - expect((await lwsBridge(fetch)(MONTH, PATCH(INSERT))).status).toBe(204); - }); -}); diff --git a/packages/solid/src/lwsBridge.ts b/packages/solid/src/lwsBridge.ts deleted file mode 100644 index 263fdcd73..000000000 --- a/packages/solid/src/lwsBridge.ts +++ /dev/null @@ -1,404 +0,0 @@ -// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors -// -// SPDX-License-Identifier: MIT - -import { isInFolder, isSameUrl } from "@solid-memo/domain/instanceLayout"; -import type { ReadLimits } from "./client/interface"; -import { linkTargets, TURTLE_FIRST } from "./client/linkHeader"; -import { limitedFetch } from "./client/readLimits"; -import { dataBlocks, parentOf, patched } from "./localPod"; -import { LWS_STORAGE_REL } from "./lwsStorage"; -import { triplesOf } from "./ntriples"; - -/** - * The bridge between the Solid Protocol requests the app makes and a - * Linked Web Storage (LWS) server that takes only what LWS asks of it - * (docs/lws.md). Three requests of the app have no LWS equivalent; each - * is first sent as it is, and only when the server refuses it, and - * says it is an LWS server, made as LWS has it: - * - * - a container's listing in Turtle (`ldp:contains`): LWS lists a - * container as `application/lws+json`, page by page; the pages are - * read and given as the Turtle listing the app reads; - * - a document or container made with PUT and `If-None-Match: *`: LWS - * makes a resource with a POST to its container, which names it, and - * each container above it first; the name asked for is sent as a - * hint (`Slug`), and a resource the server names otherwise is deleted - * and the creation refused (CREATION_REFUSED); - * - a SPARQL Update PATCH (`INSERT DATA`, `DELETE DATA`): LWS requires - * only JSON Patch; the document is read, changed here, and written - * whole with `If-Match`, its ETag as read when the PATCH had none, - * tried again when another write came first; a document not there - * yet is made as above. - * - * Every other request, and every request to a Solid server, is sent as - * it is. It sits below the fences (podFence.ts, ownPodGate.ts), which - * check each request as the app makes it; what it sends besides is to - * the same resource, the containers above it, or the pages the server - * links. It imports no library. - */ -export function lwsBridge(fetch: typeof globalThis.fetch): typeof globalThis.fetch { - return async (input, init) => { - // The app asks by URL; a Request is sent as it is. - if (typeof input !== "string" && !(input instanceof URL)) return fetch(input, init); - const url = String(input); - const method = (init?.method ?? "GET").toUpperCase(); - const headers = new Headers(init?.headers); - if (method === "GET" && url.endsWith("/")) return listing(fetch, url, init); - if (method === "PUT" && headers.get("If-None-Match") === "*") - return creation(fetch, url, init!); - if (method === "PATCH" && mediaTypeOf(headers.get("Content-Type")) === SPARQL_UPDATE) - return update(fetch, url, init!); - return fetch(input, init); - }; -} - -/** The status of a creation the server refused to make by the name asked for: it names its resources itself. */ -export const CREATION_REFUSED = 501; - -/** The status of a creation the server made by another name, which the bridge could not delete again: never tried again. */ -const CREATION_LEFT = 502; - -const LWS_JSON = "application/lws+json"; -const LWS_PROFILE = "https://www.w3.org/ns/lws/v1"; -const LWS_CONTAINER = "https://www.w3.org/ns/lws#Container"; -const LDP = "http://www.w3.org/ns/ldp#"; -const SPARQL_UPDATE = "application/sparql-update"; -/** What a server answers a request whose method or media type it does not take. */ -const NOT_TAKEN = new Set([405, 415, 501]); -/** The most pages of a listing, and items, read: a listing past them is refused, as a container past the tree budget is (containers.ts). */ -const MAX_PAGES = 200; -const MAX_ITEMS = 20_000; -/** Each page of a listing is read under these limits. */ -const PAGE_LIMITS: ReadLimits = { bytes: 4 * 1024 * 1024, quads: 0, characters: 4 * 1024 * 1024 }; -/** The most containers above a resource made, or asked whether the server is an LWS one. */ -const MAX_DEPTH = 32; -/** The most times a PATCH without `If-Match` is tried again after another write came first. */ -const MAX_TRIES = 5; - -/** A media type without its parameters, in lower case; "" for none. */ -function mediaTypeOf(contentType: string | null): string { - return (contentType ?? "").split(";")[0]!.trim().toLowerCase(); -} - -/** Whether a Content-Type names `profile` in its `profile` parameter, a space-separated list, quoted or not. */ -function hasProfile(contentType: string | null, profile: string): boolean { - const parameter = /;\s*profile\s*=\s*(?:"([^"]*)"|([^\s;]+))/i.exec(contentType ?? ""); - const profiles = parameter?.[1] ?? parameter?.[2] ?? ""; - return profiles.split(/\s+/).some((named) => named === profile); -} - -/** A response for `url`, as a server's answer is: its URL set. */ -function named(response: Response, url: string): Response { - Object.defineProperty(response, "url", { value: url }); - return response; -} - -/** - * Whether the server of `url` is an LWS server: the first of `url` and - * the containers above it that is there links its storage - * (lwsStorage.ts), as every answer of an LWS server to a GET or HEAD - * does. - */ -async function isLws( - fetch: typeof globalThis.fetch, - url: string, - signal: AbortSignal | null | undefined, -): Promise { - let at: string | null = url; - for (let depth = 0; at !== null && depth < MAX_DEPTH; depth++) { - const response = await fetch(at, { - method: "HEAD", - headers: { Accept: TURTLE_FIRST }, - signal, - }); - if (response.status !== 404) return linkTargets(response, LWS_STORAGE_REL, at).length > 0; - at = parentOf(at); - } - return false; -} - -// The listing. - -async function listing( - fetch: typeof globalThis.fetch, - url: string, - init: RequestInit | undefined, -): Promise { - const response = await fetch(url, init); - const contentType = response.headers.get("Content-Type"); - if (response.ok && mediaTypeOf(contentType) === LWS_JSON) - return listingOf(fetch, url, init, response); - // An LWS server that gives Turtle in the LWS profile (lws:items, no ldp:contains), or none, is asked for its own listing. - const lwsProfile = response.ok && hasProfile(contentType, LWS_PROFILE); - if (response.status !== 406 && !lwsProfile) return response; - if (!(await isLws(fetch, url, init?.signal))) return response; - await response.body?.cancel(); - const asked = await fetch(url, { ...init, headers: withAccept(init, LWS_JSON) }); - if (!asked.ok || mediaTypeOf(asked.headers.get("Content-Type")) !== LWS_JSON) return asked; - return listingOf(fetch, url, init, asked); -} - -function withAccept(init: RequestInit | undefined, accept: string): Headers { - const headers = new Headers(init?.headers); - headers.set("Accept", accept); - return headers; -} - -/** - * The listing of the container at `url` whose first page is `first` - * (`application/lws+json`), every page read, as the Turtle the app - * reads: the container, typed as one, and `ldp:contains` each resource - * in it. The first page's status and headers (its ETag, its links) - * are kept. A page that is not a container's listing, or past the - * limits, fails the read. - */ -async function listingOf( - fetch: typeof globalThis.fetch, - url: string, - init: RequestInit | undefined, - first: Response, -): Promise { - const pageFetch = limitedFetch(fetch, PAGE_LIMITS); - const items = new Set(); - const seen = new Set(); - let page = await limitedFetch(async () => first, PAGE_LIMITS)(url); - let at = first.url || url; - for (let pages = 1; ; pages++) { - const body: unknown = JSON.parse(await page.text()); - const listed = (body as { items?: unknown } | null)?.items; - if (!Array.isArray(listed)) throw new Error(`${at} is not the listing of a container.`); - for (const item of listed) { - const id = (item as { id?: unknown } | null)?.id; - if (typeof id !== "string") continue; - const member = new URL(id, at).href; - if (isInFolder(member, url) && !isSameUrl(member, url)) items.add(member); - } - if (items.size > MAX_ITEMS) throw new Error(`${url} lists more than ${MAX_ITEMS} resources.`); - seen.add(at); - const [next] = linkTargets(page, "next", at); - if (next === undefined) break; - if (new URL(next).origin !== new URL(url).origin || seen.has(next) || pages === MAX_PAGES) - throw new Error(`${url}'s listing goes on past what the app reads.`); - page = await pageFetch(next, { headers: { Accept: LWS_JSON }, signal: init?.signal }); - if (!page.ok) throw new Error(`${next} answered ${page.status}.`); - at = next; - } - const contains = [...items].map((member) => `<${iriOf(member)}>`).join(", "); - const turtle = `<${url}> a <${LDP}Container>, <${LDP}BasicContainer>${ - contains === "" ? "" : `;\n <${LDP}contains> ${contains}` - } .\n`; - const headers = new Headers(first.headers); - headers.set("Content-Type", "text/turtle"); - headers.delete("Content-Length"); - return named(new Response(turtle, { status: first.status, headers }), first.url || url); -} - -/** An address as a Turtle IRI: the characters an IRI may not hold, which a URL's path keeps (`|`, `^`), percent-encoded. */ -function iriOf(url: string): string { - return url.replace( - /[\u0000-\u0020<>"{}|^`\\]/g, - (c) => `%${c.charCodeAt(0).toString(16).toUpperCase().padStart(2, "0")}`, - ); -} - -// Creation. - -async function creation( - fetch: typeof globalThis.fetch, - url: string, - init: RequestInit, -): Promise { - const response = await fetch(url, init); - if (response.status !== 404 && !NOT_TAKEN.has(response.status)) return response; - const parent = parentOf(url); - if (parent === null || !(await isLws(fetch, parent, init.signal))) return response; - return post(fetch, url, init); -} - -/** - * The resource at `url` made as LWS makes one: each container above it - * that is not there made first, then a POST to its container, its name - * asked for. The answer is the POST's, its URL `url`; CREATION_REFUSED - * when the server named it otherwise (and it is deleted again). - */ -async function post( - fetch: typeof globalThis.fetch, - url: string, - init: RequestInit, - depth = 0, -): Promise { - const parent = parentOf(url)!; - const above = await containerThere(fetch, parent, init, depth); - if (above !== null) return above; - const container = url.endsWith("/"); - const headers = new Headers(init.headers); - // A creation's conditions mean nothing to a POST, and a type it links is the server's to set (LWS: server-managed). - headers.delete("If-None-Match"); - headers.delete("If-Match"); - headers.delete("Link"); - // The name as the address spells it, its escapes kept: a header's value is bytes (RFC 5023 has the server undo them). - headers.set( - "Slug", - new URL(url).pathname - .split("/") - .filter((segment) => segment !== "") - .at(-1)!, - ); - if (container) { - // A container: made with no body, its type linked, as LWS asks of the client. - headers.delete("Content-Type"); - headers.set("Link", `<${LWS_CONTAINER}>; rel="type"`); - } - const posted = await fetch(parent, { - method: "POST", - headers, - body: container ? undefined : init.body, - redirect: init.redirect, - signal: init.signal, - }); - if (!posted.ok) return posted; - const location = posted.headers.get("Location"); - const made = location === null ? null : new URL(location, parent).href; - if (made !== null && isSameUrl(made, url)) return named(posted, url); - // Only a resource the POST made in the container is deleted again: another address the answer names is no one's to delete here. - const madeHere = - made !== null && new URL(made).origin === new URL(parent).origin && parentOf(made) === parent; - const deleted = madeHere && (await fetch(made, { method: "DELETE", signal: init.signal })).ok; - return named( - deleted - ? new Response(null, { - status: CREATION_REFUSED, - statusText: "The server names its resources itself", - }) - : new Response(null, { - status: CREATION_LEFT, - statusText: "The server made the resource where it would not say, or could not delete it", - }), - url, - ); -} - -/** Null when the container at `url` is there, or once it is made (post); else the answer that says why it is not. */ -async function containerThere( - fetch: typeof globalThis.fetch, - url: string, - init: RequestInit, - depth: number, -): Promise { - const response = await fetch(url, { - method: "HEAD", - headers: { Accept: TURTLE_FIRST }, - signal: init.signal, - }); - if (response.ok) return null; - if (response.status !== 404 || depth >= MAX_DEPTH || parentOf(url) === null) return response; - const made = await post( - fetch, - url, - { method: "PUT", signal: init.signal, redirect: init.redirect }, - depth + 1, - ); - return made.ok ? null : made; -} - -// The update. - -async function update( - fetch: typeof globalThis.fetch, - url: string, - init: RequestInit, -): Promise { - const refused = await fetch(url, init); - if (refused.ok || (refused.status !== 404 && !NOT_TAKEN.has(refused.status))) return refused; - if (!(await isLws(fetch, url, init.signal))) return refused; - const edit: Edit = { - url, - init, - sparql: String(init.body), - condition: new Headers(init.headers).get("If-Match"), - refused, - }; - let last: Try = { answer: refused, again: false }; - for (let tries = 0; tries < MAX_TRIES && (tries === 0 || last.again); tries++) - last = await updateOnce(fetch, edit); - return last.answer; -} - -/** A SPARQL Update the server refused: the document, the request, the update, its If-Match, and the refusal. */ -interface Edit { - url: string; - init: RequestInit; - sparql: string; - condition: string | null; - refused: Response; -} - -/** The answer to one try of an update, and whether to try again: another write came first to an update with no If-Match. */ -interface Try { - answer: Response; - again: boolean; -} - -const conflict = (url: string) => named(new Response(null, { status: 409 }), url); - -/** The edit made once: the document read, changed here, and written whole with If-Match. */ -async function updateOnce(fetch: typeof globalThis.fetch, edit: Edit): Promise { - const { url, init, sparql, condition } = edit; - const current = await fetch(url, { - headers: { Accept: "text/turtle" }, - cache: "no-store", - signal: init.signal, - }); - if (current.status === 404) return madeBy(fetch, edit); - if (!current.ok) return { answer: current, again: false }; - // A weak ETag never matches If-Match (RFC 9110): a document with none strong has no version to write against. - const etag = current.headers.get("ETag"); - // An edit of the document as read, which has changed since: another write came first, as the PUT would say. - if (condition !== null && etag !== null && etag !== condition) - return { answer: named(new Response(null, { status: 412 }), url), again: false }; - const version = condition ?? (etag === null || etag.startsWith("W/") ? null : etag); - const read = new Set(await triplesOf(url, await current.text())); - const triples = await patched(url, [...read], sparql); - // An update the app does not make, or a document with no version to write it against, is not written. - if (triples === null || version === null) return { answer: edit.refused, again: false }; - // A triple to delete that is not there fails the update, as a Solid server's PATCH does (409). - if ([...(await deletionsOf(url, sparql))].some((triple) => !read.has(triple))) - return { answer: conflict(url), again: false }; - const written = await fetch(url, { - method: "PUT", - headers: { "Content-Type": "text/turtle", "If-Match": version }, - body: `${triples.join("\n")}\n`, - redirect: init.redirect, - signal: init.signal, - }); - return { answer: written, again: written.status === 412 && condition === null }; -} - -/** The edit of a document not there yet: the document made with what it inserts, unless it was an edit of one as read. */ -async function madeBy(fetch: typeof globalThis.fetch, edit: Edit): Promise { - const { url, init, sparql } = edit; - // An edit of a document as read finds none: another write came first. - if (edit.condition !== null) - return { answer: named(new Response(null, { status: 412 }), url), again: false }; - const triples = await patched(url, [], sparql); - if (triples === null) return { answer: edit.refused, again: false }; - if ((await deletionsOf(url, sparql)).size > 0) return { answer: conflict(url), again: false }; - const made = await post(fetch, url, { - ...init, - method: "PUT", - headers: { "Content-Type": "text/turtle" }, - body: `${triples.join("\n")}\n`, - }); - // Another tab made it first, by the same name: its document is changed instead. - return { answer: made, again: made.status === CREATION_REFUSED }; -} - -/** The triples an update's `DELETE DATA` blocks name, as N-Triples; an update `patched` applies only. */ -async function deletionsOf(url: string, sparql: string): Promise> { - const deletions = new Set(); - for (const { operation, data } of dataBlocks(sparql)!) - if (operation === "DELETE") - for (const triple of await triplesOf(url, data)) deletions.add(triple); - return deletions; -} diff --git a/packages/solid/src/testing/lwsPod.ts b/packages/solid/src/testing/lwsPod.ts deleted file mode 100644 index fe2d5f54f..000000000 --- a/packages/solid/src/testing/lwsPod.ts +++ /dev/null @@ -1,192 +0,0 @@ -// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors -// -// SPDX-License-Identifier: MIT - -/** - * A Linked Web Storage server, in memory, that takes only what LWS - * Protocol 1.0 asks of a server (docs/lws.md), for the tests of the - * bridge (lwsBridge.ts): a container is listed only as - * `application/lws+json` (Turtle refused with 406, or given in the LWS - * profile), page by page; a resource is made only by a POST to its - * container, which names it from the `Slug` it is given (or, when told - * so, by a name of its own); PUT replaces a resource that is there and - * answers 404 for one that is not; PATCH is refused (415), as this one - * takes no JSON Patch; every answer to a GET or HEAD links the storage. - */ -export interface LwsPodOptions { - /** The storage's URI, its root container's too: a URL ending in a slash. */ - root: string; - /** The most items on a page of a listing. */ - pageSize?: number; - /** Turtle in the LWS profile for a container's Turtle, not 406. */ - profileTurtle?: boolean; - /** Names a resource made by POST itself, the Slug left aside. */ - ownNames?: boolean; -} - -export interface LwsPod { - fetch: typeof globalThis.fetch; - /** The documents, by URL, and the containers (their URL ends in a slash, their body ""). */ - resources: Map; - /** Every request, as "METHOD url". */ - requests: string[]; -} - -const STORAGE_REL = "https://www.w3.org/ns/lws#storage"; - -export function lwsPod({ - root, - pageSize = 1000, - profileTurtle = false, - ownNames = false, -}: LwsPodOptions): LwsPod { - const resources = new Map(); - const requests: string[] = []; - let version = 0; - let named = 0; - const etag = () => `"${++version}"`; - /** A container's ETag changes with its members, as a listing's does. */ - const touch = (container: string) => { - const there = resources.get(container); - if (there !== undefined) resources.set(container, { ...there, etag: etag() }); - }; - resources.set(root, { body: "", type: "", etag: etag() }); - - const parentOf = (url: string) => { - const trimmed = url.endsWith("/") ? url.slice(0, -1) : url; - return trimmed.slice(0, trimmed.lastIndexOf("/") + 1); - }; - const members = (container: string) => - [...resources.keys()].filter((url) => url !== container && parentOf(url) === container).sort(); - const answer = ( - status: number, - body: string | null, - headers: Record, - url: string, - ) => { - const response = new Response(status === 204 || status === 304 ? null : body, { - status, - headers: { Link: `<${root}>; rel="${STORAGE_REL}"`, ...headers }, - }); - Object.defineProperty(response, "url", { value: url }); - return response; - }; - - /** What a request names: the resource (without a page's query), the address asked, its headers and body. */ - interface Asked { - url: string; - href: string; - method: string; - headers: Headers; - body: string; - } - - /** A container's listing, as LWS has it: a page of its members, linked to the next. */ - const listing = ({ url, href, method }: Asked, etagOf: string) => { - const all = members(url); - const page = Number(new URL(href).searchParams.get("page") ?? "0"); - const items = all.slice(page * pageSize, (page + 1) * pageSize).map((id) => { - const member = resources.get(id)!; - return id.endsWith("/") - ? { id, type: "Container" } - : { id, type: "DataResource", format: member.type, size: member.body.length }; - }); - const links = [`<${root}>; rel="${STORAGE_REL}"`]; - if (all.length > pageSize) links.push(`<${url}?page=0>; rel="first"`); - if ((page + 1) * pageSize < all.length) links.push(`<${url}?page=${page + 1}>; rel="next"`); - const body = JSON.stringify({ - "@context": "https://www.w3.org/ns/lws/v1", - id: url, - type: "Container", - totalItems: all.length, - items, - }); - return answer( - 200, - method === "HEAD" ? null : body, - { "Content-Type": "application/lws+json", ETag: etagOf, Link: links.join(", ") }, - href, - ); - }; - - const read = (asked: Asked) => { - const { url, href, method, headers } = asked; - const resource = resources.get(url); - if (resource === undefined) return answer(404, null, {}, href); - if (headers.get("If-None-Match") === resource.etag) - return answer(304, null, { ETag: resource.etag }, href); - const body = method === "HEAD" ? null : resource.body; - if (!url.endsWith("/")) - return answer(200, body, { "Content-Type": resource.type, ETag: resource.etag }, href); - const accept = headers.get("Accept") ?? "*/*"; - if (accept.includes("application/lws+json") || accept.includes("*/*")) - return listing(asked, resource.etag); - if (!profileTurtle) return answer(406, null, {}, href); - const profiled = method === "HEAD" ? null : `<> a .`; - return answer( - 200, - profiled, - { - "Content-Type": 'text/turtle; profile="https://www.w3.org/ns/lws/v1"', - ETag: resource.etag, - }, - href, - ); - }; - - const replace = ({ url, href, headers, body }: Asked) => { - const resource = resources.get(url); - if (resource === undefined) return answer(404, null, {}, href); - const ifMatch = headers.get("If-Match"); - if (headers.get("If-None-Match") === "*" || (ifMatch !== null && ifMatch !== resource.etag)) - return answer(412, null, {}, href); - resources.set(url, { body, type: headers.get("Content-Type") ?? "", etag: etag() }); - return answer(204, null, { ETag: resources.get(url)!.etag }, href); - }; - - const create = ({ url, href, headers, body }: Asked) => { - if (!resources.has(url) || !url.endsWith("/")) return answer(404, null, {}, href); - const container = (headers.get("Link") ?? "").includes("https://www.w3.org/ns/lws#Container"); - const slug = headers.get("Slug"); - const name = - ownNames || slug === null ? `r${++named}` : encodeURIComponent(decodeURIComponent(slug)); - const end = container ? "/" : ""; - const made = resources.has(`${url}${name}${end}`) - ? `${url}${name}-${++named}${end}` - : `${url}${name}${end}`; - resources.set(made, { - body: container ? "" : body, - type: container ? "" : (headers.get("Content-Type") ?? ""), - etag: etag(), - }); - touch(url); - return answer(201, null, { Location: made, ETag: resources.get(made)!.etag }, href); - }; - - const remove = ({ url, href }: Asked) => { - if (!resources.has(url)) return answer(404, null, {}, href); - if (url.endsWith("/") && members(url).length > 0) return answer(409, null, {}, href); - resources.delete(url); - touch(parentOf(url)); - return answer(204, null, {}, href); - }; - - const fetch: typeof globalThis.fetch = async (input, init) => { - const href = String(input); - const method = init?.method ?? "GET"; - requests.push(`${method} ${href}`); - const asked: Asked = { - url: href.split("?")[0]!, - href, - method, - headers: new Headers(init?.headers), - body: String(init?.body ?? ""), - }; - if (method === "GET" || method === "HEAD") return read(asked); - if (method === "PUT") return replace(asked); - if (method === "POST") return create(asked); - if (method === "DELETE") return remove(asked); - return answer(method === "PATCH" ? 415 : 405, null, {}, href); - }; - return { fetch, resources, requests }; -} From a419b03248a1222245af5f94b5fba05544a1cd34 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sat, 7 Nov 2026 17:21:41 +0100 Subject: [PATCH 14/19] test(journeys): read a deck's cards document relative to the catalog The pod-safety journey that keeps another app's data took the cards document's address from the catalog's Turtle as written. CSS with Linked Web Storage writes it relative to the catalog, so the journey wrote to a relative path and got a 404. The address is now resolved against the catalog, as a Turtle parser would. Refs: #594 --- e2e/journeys/journeys/pod-safety.journey.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/e2e/journeys/journeys/pod-safety.journey.ts b/e2e/journeys/journeys/pod-safety.journey.ts index 14a15d0f3..b0bd4afa0 100644 --- a/e2e/journeys/journeys/pod-safety.journey.ts +++ b/e2e/journeys/journeys/pod-safety.journey.ts @@ -518,7 +518,8 @@ test("deleting an instance keeps another app's data in a deck's document, and sa "04 · Another app keeps data of its own in the deck's document", async () => { const said = /v1\.ttl#cardsDocument>\s+<([^>]+)>/.exec(await user.read(catalog)); - cards = said![1]!; + // Relative to the catalog where the server writes its Turtle so (CSS with Linked Web Storage). + cards = new URL(said![1]!, catalog).href; const served = (await user.read(cards)).replace(/^200 /, ""); await user.put(cards, `${served}\n<#note-${runId}> "mine" .\n`); }, From 982b60c563bb0c0f5fb4c28adeb3f3e4e5ccf07a Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sat, 7 Nov 2026 17:21:52 +0100 Subject: [PATCH 15/19] test(journeys): run the user journeys against the CSS LWS servers The journeys ran only on Community Solid Server 7, so the app's sign-in to a Linked Web Storage storage in a browser was tried by hand alone. JOURNEY_SERVER=css-lws or css-lws-only now starts that server's signed-in instance, the fork's own configuration behind TLS, from the image e2e/pod builds, in place of the compose file kept for the hand runs. Where a journey reads or writes the pod itself, the harness now gets a token as the app does there: it logs the account in on the authorization-code flow and exchanges the ID token for the storage's. The Solid-OIDC token client credentials give is refused by an LWS storage, which failed the journeys that write as another app. On a newly started server, token exchanges made side by side each made a signing key, and the server refused every token but one key's. The harness reads the server's keys once before the journeys start, which makes the one key. Refs: #594 --- docs/lws.md | 30 ++-- docs/testing.md | 9 ++ e2e/journeys/css-lws/compose.yml | 44 ------ e2e/journeys/harness/cssServer.ts | 43 ++++-- e2e/journeys/harness/journeyServer.test.ts | 37 +++++ e2e/journeys/harness/journeyServer.ts | 56 ++++++++ e2e/journeys/harness/otherApp.ts | 8 +- e2e/journeys/harness/podDump.ts | 28 +--- e2e/journeys/harness/podToken.ts | 145 ++++++++++++++++++++ e2e/journeys/journeys/pod-safety.journey.ts | 7 +- 10 files changed, 312 insertions(+), 95 deletions(-) delete mode 100644 e2e/journeys/css-lws/compose.yml create mode 100644 e2e/journeys/harness/journeyServer.test.ts create mode 100644 e2e/journeys/harness/journeyServer.ts create mode 100644 e2e/journeys/harness/podToken.ts diff --git a/docs/lws.md b/docs/lws.md index fb37c6d8c..5b798015a 100644 --- a/docs/lws.md +++ b/docs/lws.md @@ -76,19 +76,27 @@ keeps one token for each storage. Inrupt's sign-in client gives the app the ID token only in the address that ends the session, so a build that signs in to an LWS storage is made with `SOLID_CLIENT=solid-memo-client`. -## Trying it by hand +## How it's tested [The signed-in tests](testing.md#the-signed-in-tests) sign in to an LWS -storage without a browser. The login in a browser is tried by hand, with -the user journeys -([e2e/journeys/css-lws/compose.yml](../e2e/journeys/css-lws/compose.yml)). -The fork's image, which `npm run servers -w @solid-memo/e2e-pod -- -css-lws` builds, runs its own `config/lws.json` behind the journeys' -TLS, each account's pod a storage of its own. The app is built with -`SOLID_CLIENT=solid-memo-client`, and `JOURNEY_SERVER_URL` names the -server. A journey that writes to the pod as another app -(`harness/otherApp.ts`) fails there: its Solid-OIDC token is no LWS -storage's. +storage without a browser. [The user journeys](testing.md#user-journeys) +sign in with one, in Chromium, on both servers. `JOURNEY_SERVER=css-lws` +or `css-lws-only` runs them against that server's signed-in instance: +the fork's own configuration behind TLS, with a storage for each pod. +The app is built with `SOLID_CLIENT=solid-memo-client`. The Interop +workflow runs them. + +Some journeys read or write the pod themselves, as another app or to +check what the app left. There the harness does what the app does: it +logs the account in on the authorization-code flow and exchanges its ID +token for the storage's token (`harness/podToken.ts`). The Solid-OIDC +token that client credentials give is no LWS storage's. + +**Why the harness reads the server's keys first.** On a newly started +server, token exchanges made side by side can each make the key the +server signs with. The server then refuses every token but those of the +key it kept ("signature verification failed"). One read of its keys +(`/.well-known/lws/jwks`) before the journeys start makes the one key. The fork's code is where this belongs fixed. ## What LWS can't give the app yet diff --git a/docs/testing.md b/docs/testing.md index 6eb83fe4f..69b4f1678 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -439,6 +439,15 @@ need Docker and the browsers Firefox and WebKit, with their system libraries). CI runs them as a gate ([below](#deterministic-and-timing-dependent-jobs)). +`JOURNEY_SERVER` names another server to run them against: `css-lws` or +`css-lws-only`, the advisory servers above with Linked Web Storage, as +their signed-in instances behind TLS +([harness/journeyServer.ts](../e2e/journeys/harness/journeyServer.ts)), +from the image e2e/pod builds. The app signs in to them only when built +with `SOLID_CLIENT=solid-memo-client` ([lws.md](lws.md#how-its-tested)). +The Interop workflow runs Chromium's journeys on both and reports them, +as it does the pod tests: they gate nothing. + Further journeys go through: - a deck added by link whose cards name pictures on its author's diff --git a/e2e/journeys/css-lws/compose.yml b/e2e/journeys/css-lws/compose.yml deleted file mode 100644 index ca5be475f..000000000 --- a/e2e/journeys/css-lws/compose.yml +++ /dev/null @@ -1,44 +0,0 @@ -# SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors -# -# SPDX-License-Identifier: MIT - -# The Community Solid Server 8 with Linked Web Storage alone (the fork's -# own config/lws.json: files, a storage for each pod), from the image -# e2e/pod/servers/css-lws/ builds (`npm run servers -w @solid-memo/e2e-pod -# -- css-lws`), behind css/'s TLS, for a contributor to run the journeys -# on by hand (docs/lws.md): `E2E_PORT= docker compose -f -# e2e/journeys/css-lws/compose.yml up -d`, then -# `JOURNEY_SERVER_URL=https://127.0.0.1:/ npm run journeys` with the -# app built with `SOLID_CLIENT=solid-memo-client`. CI runs none. -services: - tls: - image: caddy:2.11.7@sha256:36c24ad91a7ff7bc4c87fe22bfa8df3c4b18da0773d73fc12ee2c52b98c0bb7e - command: ["caddy", "run", "--config", "/etc/caddy/Caddyfile", "--adapter", "caddyfile"] - environment: { E2E_PORT: "${E2E_PORT:?}" } - volumes: ["../css/Caddyfile:/etc/caddy/Caddyfile:ro"] - tmpfs: ["/data:uid=1000,gid=1000", "/config:uid=1000,gid=1000"] - ports: ["127.0.0.1:${E2E_PORT:?}:${E2E_PORT:?}"] - user: "1000:1000" - cap_drop: [ALL] - # Its binary carries this file capability, without which it cannot run. - cap_add: [NET_BIND_SERVICE] - security_opt: ["no-new-privileges:true"] - mem_limit: 256m - pids_limit: 256 - server: - image: localhost/solid-memo-e2e-css-lws:local - pull_policy: never - command: ["-c", "config/lws.json", "-f", "/data", "-b", "https://127.0.0.1:${E2E_PORT:?}/", "-l", "info"] - network_mode: "service:tls" - depends_on: [tls] - # Caddy's CA is made as it starts, and this server is thrown away after - # the run: it trusts whatever certificate its own requests meet, the - # LWS token endpoint's discovery of its own OpenID provider among them. - environment: { NODE_TLS_REJECT_UNAUTHORIZED: "0" } - tmpfs: ["/data:uid=1000,gid=1000,mode=0700"] - user: node - init: true - cap_drop: [ALL] - security_opt: ["no-new-privileges:true"] - mem_limit: 1g - pids_limit: 512 diff --git a/e2e/journeys/harness/cssServer.ts b/e2e/journeys/harness/cssServer.ts index 7a19e8746..6aa3c84b0 100644 --- a/e2e/journeys/harness/cssServer.ts +++ b/e2e/journeys/harness/cssServer.ts @@ -10,15 +10,19 @@ import { createServer } from "node:net"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { request } from "@playwright/test"; +import { journeyServer } from "./journeyServer.ts"; /** - * The Community Solid Server 7 the journeys log in to and keep their pods - * on: css/compose.yml, behind TLS at https://127.0.0.1:/, one per - * run. Patterned on e2e/pod/servers.ts, which runs its servers over http. + * The Solid server the journeys log in to and keep their pods on, behind + * TLS at https://127.0.0.1:/, one per run: the Community Solid + * Server 7 of css/compose.yml, or the one JOURNEY_SERVER names + * (journeyServer.ts). Patterned on e2e/pod/servers.ts, which runs its + * servers over http. */ const HERE = join(import.meta.dirname, ".."); -const COMPOSE_FILE = join(HERE, "css", "compose.yml"); +export const SERVER = journeyServer(process.env.JOURNEY_SERVER, HERE); +const COMPOSE_FILE = SERVER.compose; export const LOGS = join(HERE, "logs"); export const CSS_LOG = join(LOGS, "css.log"); /** Which checkout the containers belong to, so two worktrees' runs keep apart. */ @@ -34,8 +38,15 @@ export interface StartedCss { stop: () => Promise; } -/** Pulls the images unless they are here. */ +/** + * Pulls the images unless they are here; a Linked Web Storage server's, + * which no registry has, e2e/pod builds (or tags, from its cache). + */ export async function prepare(): Promise { + if (SERVER.lws) { + await run(process.execPath, [join(HERE, "..", "pod", "servers.ts"), "prepare", SERVER.id], {}); + return; + } await docker( [ "compose", @@ -97,6 +108,10 @@ export async function startCss(): Promise { const url = `https://127.0.0.1:${env.E2E_PORT}/`; try { await untilUp(`${url}.well-known/openid-configuration`); + // Its first token exchanges, side by side, may each make a signing + // key, and all but one token are then refused: reading its keys + // first makes the one (docs/lws.md#how-its-tested). + if (SERVER.lws) await untilUp(`${url}.well-known/lws/jwks`); } catch (error) { await stop().catch(() => {}); throw error; @@ -140,8 +155,18 @@ function follow(project: string, env: Record): ChildProcess { /** Runs docker, resolving with what it printed; rejecting with it if it fails. */ function docker(args: string[], env: Record, cwd = HERE): Promise { + return run("docker", args, env, cwd); +} + +/** Runs `command`, resolving with what it printed; rejecting with it if it fails. */ +function run( + command: string, + args: string[], + env: Record, + cwd = HERE, +): Promise { return new Promise((resolve, reject) => { - const child = spawn("docker", args, { + const child = spawn(command, args, { cwd, env: { ...process.env, ...env }, stdio: ["ignore", "pipe", "pipe"], @@ -152,14 +177,14 @@ function docker(args: string[], env: Record, cwd = HERE): Promis child.on("error", (error) => reject( new Error( - `docker could not be run (${error.message}); the journeys start their Solid server in it (docs/testing.md).`, + `${command} could not be run (${error.message}); the journeys start their Solid server in Docker (docs/testing.md).`, ), ), ); child.on("close", (code) => code === 0 ? resolve(out) - : reject(new Error(`docker ${args.join(" ")} failed (${code}):\n${out}`)), + : reject(new Error(`${command} ${args.join(" ")} failed (${code}):\n${out}`)), ); }); } @@ -213,7 +238,7 @@ if (import.meta.main) { for (const signal of ["SIGINT", "SIGTERM"] as const) process.once(signal, () => void stop().finally(() => process.exit(0))); console.log( - `Community Solid Server 7 at ${url} (its log: ${CSS_LOG}). Run the journeys against it with\n JOURNEY_SERVER_URL=${url} npm run journeys\nCtrl-C stops it.`, + `${SERVER.name} at ${url} (its log: ${CSS_LOG}). Run the journeys against it with\n JOURNEY_SERVER_URL=${url} npm run journeys\nCtrl-C stops it.`, ); await new Promise(() => {}); } else { diff --git a/e2e/journeys/harness/journeyServer.test.ts b/e2e/journeys/harness/journeyServer.test.ts new file mode 100644 index 000000000..8d0bc14bb --- /dev/null +++ b/e2e/journeys/harness/journeyServer.test.ts @@ -0,0 +1,37 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { describe, expect, it } from "vitest"; +import { journeyServer } from "./journeyServer.ts"; + +describe("journeyServer", () => { + it("is Community Solid Server 7 when JOURNEY_SERVER names none", () => { + for (const given of [undefined, ""]) + expect(journeyServer(given, "/e2e/journeys")).toEqual({ + id: "css-7", + name: "Community Solid Server 7", + compose: "/e2e/journeys/css/compose.yml", + lws: false, + }); + }); + + it("runs a Linked Web Storage server's signed-in instance, which e2e/pod builds", () => { + expect(journeyServer("css-lws-only", "/e2e/journeys")).toEqual({ + id: "css-lws-only", + name: "Community Solid Server 8 with Linked Web Storage alone", + compose: "/e2e/pod/servers/css-lws-only/signed-in.yml", + lws: true, + }); + expect(journeyServer("css-lws", "/e2e/journeys").compose).toBe( + "/e2e/pod/servers/css-lws/signed-in.yml", + ); + }); + + it("refuses a server it does not know, naming those it does", () => { + expect(() => journeyServer("nss", "/e2e/journeys")).toThrow( + 'JOURNEY_SERVER="nss" names no server the journeys run against: leave it unset, or set it to "css-7", "css-lws", "css-lws-only".', + ); + expect(() => journeyServer("toString", "/e2e/journeys")).toThrow("names no server"); + }); +}); diff --git a/e2e/journeys/harness/journeyServer.ts b/e2e/journeys/harness/journeyServer.ts new file mode 100644 index 000000000..6d4a6609a --- /dev/null +++ b/e2e/journeys/harness/journeyServer.ts @@ -0,0 +1,56 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { join } from "node:path"; + +/** + * The Solid servers the journeys run against, which JOURNEY_SERVER + * names (docs/testing.md#user-journeys): the Community Solid Server 7 + * they gate on, first, and the advisory ones with Linked Web Storage, + * which the Interop workflow runs them on. + */ +const SERVERS = { + "css-7": { + name: "Community Solid Server 7", + compose: "css/compose.yml", + lws: false, + }, + // Their signed-in instances, which the pod tests sign in to: the fork's own configurations, behind TLS. + "css-lws": { + name: "Community Solid Server 8 with Linked Web Storage", + compose: "../pod/servers/css-lws/signed-in.yml", + lws: true, + }, + "css-lws-only": { + name: "Community Solid Server 8 with Linked Web Storage alone", + compose: "../pod/servers/css-lws-only/signed-in.yml", + lws: true, + }, +} as const; + +export type JourneyServerId = keyof typeof SERVERS; + +export interface JourneyServer { + id: JourneyServerId; + name: string; + /** Its compose file, under `root` (e2e/journeys). */ + compose: string; + /** Whether it speaks Linked Web Storage: its image is built by e2e/pod, and its token signer is woken first. */ + lws: boolean; +} + +/** The server `given` names (JOURNEY_SERVER), Community Solid Server 7 when it names none. */ +export function journeyServer(given: string | undefined, root: string): JourneyServer { + const id = given === undefined || given === "" ? "css-7" : given; + if (!Object.hasOwn(SERVERS, id)) + throw new Error( + `JOURNEY_SERVER="${id}" names no server the journeys run against: leave it unset, or set it to ${Object.keys( + SERVERS, + ) + .map((one) => `"${one}"`) + .join(", ")}.`, + ); + const server = SERVERS[id as JourneyServerId]; + return { id: id as JourneyServerId, ...server, compose: join(root, server.compose) }; +} diff --git a/e2e/journeys/harness/otherApp.ts b/e2e/journeys/harness/otherApp.ts index 79a7b570f..aebb1da98 100644 --- a/e2e/journeys/harness/otherApp.ts +++ b/e2e/journeys/harness/otherApp.ts @@ -4,12 +4,12 @@ import type { APIRequestContext } from "@playwright/test"; import type { CssAccount } from "./cssAccount.ts"; -import { bearer } from "./podDump.ts"; +import { podToken } from "./podToken.ts"; /** * An app other than Solid Memo, acting for the journey's WebID in its - * pod with the account's client credentials, not through the browser: - * what puts files of its own in a folder Solid Memo also uses, which + * pod with a token of the account's own (podToken.ts), not through the + * browser: what puts files of its own in a folder Solid Memo also uses, which * Solid Memo must then leave be (docs/data-model.md "Deleting an * instance"). */ @@ -24,7 +24,7 @@ export interface OtherApp { export function otherApp(api: APIRequestContext, server: string, account: CssAccount): OtherApp { let token: Promise | undefined; const headers = async () => ({ - authorization: `Bearer ${await (token ??= bearer(api, server, account))}`, + authorization: `Bearer ${await (token ??= podToken(api, server, account))}`, }); return { async put(url, body, contentType) { diff --git a/e2e/journeys/harness/podDump.ts b/e2e/journeys/harness/podDump.ts index 3b7ddcc52..75ac477c9 100644 --- a/e2e/journeys/harness/podDump.ts +++ b/e2e/journeys/harness/podDump.ts @@ -4,12 +4,12 @@ import type { APIRequestContext } from "@playwright/test"; import type { CssAccount } from "./cssAccount.ts"; +import { podToken } from "./podToken.ts"; /** * Everything in a journey's pod, as Turtle: what the app had written when - * the journey failed, attached to its report. Read with the account's - * client credentials (a Bearer token from the server's token endpoint), - * not through the browser, whose page may be the thing that broke. + * the journey failed, attached to its report. Read with a token of the + * account's own (podToken.ts), not through the browser, whose page may be the thing that broke. */ const LDP_CONTAINS = "http://www.w3.org/ns/ldp#contains"; @@ -39,33 +39,13 @@ export function containedIn(jsonld: unknown, container: string): string[] { return [...found].sort(); } -/** A token for the account's WebID, by its client credentials. */ -export async function bearer( - api: APIRequestContext, - server: string, - account: CssAccount, -): Promise { - const basic = Buffer.from( - `${encodeURIComponent(account.client.id)}:${encodeURIComponent(account.client.secret)}`, - ).toString("base64"); - const response = await api.post(new URL(".oidc/token", server).href, { - headers: { authorization: `Basic ${basic}` }, - form: { grant_type: "client_credentials", scope: "webid" }, - }); - if (!response.ok()) - throw new Error( - `The token endpoint refused the client credentials: ${response.status()} ${await response.text()}`, - ); - return ((await response.json()) as { access_token: string }).access_token; -} - /** The pod's documents, each under a `# ` line, containers first listing what they hold. */ export async function dumpPod( api: APIRequestContext, server: string, account: CssAccount, ): Promise { - const headers = { authorization: `Bearer ${await bearer(api, server, account)}` }; + const headers = { authorization: `Bearer ${await podToken(api, server, account)}` }; const out: string[] = []; const queue = [account.pod]; const seen = new Set(queue); diff --git a/e2e/journeys/harness/podToken.ts b/e2e/journeys/harness/podToken.ts new file mode 100644 index 000000000..4bf909a87 --- /dev/null +++ b/e2e/journeys/harness/podToken.ts @@ -0,0 +1,145 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { createHash, randomUUID } from "node:crypto"; +import type { APIRequestContext, APIResponse } from "@playwright/test"; +import type { CssAccount } from "./cssAccount.ts"; +import { SERVER } from "./cssServer.ts"; + +/** + * A Bearer token for the account's WebID in its pod, which the harness + * reads and writes it with outside the browser (podDump.ts, otherApp.ts, + * pod-safety's documents of the user's own). On a Community Solid Server, + * from the account's client credentials. A Linked Web Storage server + * takes only a token its own authorization server issues for an OpenID + * Connect ID token, which client credentials don't give: there the + * account logs in on the authorization-code flow, and its ID token is + * exchanged for one, as the app's client does (docs/lws.md#sign-in). + */ +export function podToken( + api: APIRequestContext, + server: string, + account: CssAccount, +): Promise { + return SERVER.lws ? lwsToken(api, server, account) : clientCredentialsToken(api, server, account); +} + +/** A token for the account's WebID, by its client credentials. */ +async function clientCredentialsToken( + api: APIRequestContext, + server: string, + account: CssAccount, +): Promise { + const basic = Buffer.from( + `${encodeURIComponent(account.client.id)}:${encodeURIComponent(account.client.secret)}`, + ).toString("base64"); + const { access_token } = await answered<{ access_token: string }>( + "The token endpoint refused the client credentials", + await api.post(new URL(".oidc/token", server).href, { + headers: { authorization: `Basic ${basic}` }, + form: { grant_type: "client_credentials", scope: "webid" }, + }), + ); + return access_token; +} + +/** Where the login is sent back to: never reached, its code read from the address. */ +const REDIRECT_URI = "https://journeys.example/callback"; + +/** + * The storage's token: the account logged in as the server's pages would + * (a client registered, the login, the WebID picked, the consent, the + * code redeemed), its ID token exchanged for the pod. The login's cookies + * stay in `api`, which is never the browser's. + */ +async function lwsToken(api: APIRequestContext, server: string, account: CssAccount) { + const provider = await answered<{ + registration_endpoint: string; + authorization_endpoint: string; + token_endpoint: string; + }>( + "Reading the provider", + await api.get(new URL(".well-known/openid-configuration", server).href), + ); + const { client_id: clientId } = await answered<{ client_id: string }>( + "Registering a client", + await api.post(provider.registration_endpoint, { + data: { + redirect_uris: [REDIRECT_URI], + response_types: ["code"], + grant_types: ["authorization_code"], + token_endpoint_auth_method: "none", + application_type: "web", + scope: "openid webid", + }, + }), + ); + const verifier = randomUUID() + randomUUID(); + const follow = async (start: string) => { + let next = new URL(start, server); + for (let hop = 0; hop < 8; hop++) { + if (next.href.startsWith(REDIRECT_URI) || next.pathname.startsWith("/.account/")) return next; + const response = await api.get(next.href, { maxRedirects: 0 }); + const location = response.headers().location; + if (location === undefined) + throw new Error(`The login stopped at ${next.pathname}: ${response.status()}`); + next = new URL(location, next); + } + throw new Error(`The login never came back, ending at ${next.href}.`); + }; + const step = async (path: string, data: unknown) => + follow( + ( + await answered<{ location: string }>( + `Answering ${path}`, + await api.post(new URL(`.account/${path}`, server).href, { data }), + ) + ).location, + ); + const authorize = new URL(provider.authorization_endpoint); + authorize.search = new URLSearchParams({ + response_type: "code", + client_id: clientId, + redirect_uri: REDIRECT_URI, + scope: "openid webid", + state: randomUUID(), + nonce: randomUUID(), + code_challenge: createHash("sha256").update(verifier).digest("base64url"), + code_challenge_method: "S256", + }).toString(); + await follow(authorize.href); + await step("login/password/", { email: account.email, password: account.password }); + await step("oidc/pick-webid/", { webId: account.webId }); + const back = await step("oidc/consent/", { remember: false }); + const { id_token: idToken } = await answered<{ id_token: string }>( + "Redeeming the code", + await api.post(provider.token_endpoint, { + form: { + grant_type: "authorization_code", + client_id: clientId, + code: back.searchParams.get("code") ?? "", + redirect_uri: REDIRECT_URI, + code_verifier: verifier, + }, + }), + ); + const { access_token } = await answered<{ access_token: string }>( + "Exchanging the ID token", + await api.post(new URL(".well-known/lws/token", server).href, { + form: { + grant_type: "urn:ietf:params:oauth:grant-type:token-exchange", + resource: account.pod, + subject_token: idToken, + subject_token_type: "urn:ietf:params:oauth:token-type:id_token", + }, + }), + ); + return access_token; +} + +/** The JSON a request answered, or an error saying what was refused. */ +async function answered(what: string, response: APIResponse): Promise { + if (!response.ok()) throw new Error(`${what}: ${response.status()} ${await response.text()}`); + return (await response.json()) as T; +} diff --git a/e2e/journeys/journeys/pod-safety.journey.ts b/e2e/journeys/journeys/pod-safety.journey.ts index b0bd4afa0..dbd64177f 100644 --- a/e2e/journeys/journeys/pod-safety.journey.ts +++ b/e2e/journeys/journeys/pod-safety.journey.ts @@ -17,7 +17,8 @@ import { writeDraftWithUnsupportedChapter, } from "../flows/podDocuments.ts"; import { createAccount, type CssAccount } from "../harness/cssAccount.ts"; -import { bearer, containedIn } from "../harness/podDump.ts"; +import { containedIn } from "../harness/podDump.ts"; +import { podToken } from "../harness/podToken.ts"; import type { App } from "../pages/App.ts"; import { BRIGHTEST_STARS } from "../pages/Library.ts"; @@ -32,7 +33,7 @@ import { BRIGHTEST_STARS } from "../pages/Library.ts"; * the same status and content, and no document added. */ -/** A user of the journey's Solid server, making plain requests as themselves, with their client credentials. */ +/** A user of the journey's Solid server, making plain requests as themselves, with a token of their own (podToken.ts). */ interface PodUser { account: CssAccount; /** Writes the document at `url`, made or replaced. */ @@ -46,7 +47,7 @@ interface PodUser { function podUser(api: APIRequestContext, server: string, account: CssAccount): PodUser { let token: Promise | undefined; const headers = async () => ({ - authorization: `Bearer ${await (token ??= bearer(api, server, account))}`, + authorization: `Bearer ${await (token ??= podToken(api, server, account))}`, }); return { account, From 23d78180d9f9c373a41d5bffa3120bbe323c5fce Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sat, 7 Nov 2026 17:21:58 +0100 Subject: [PATCH 16/19] ci(journeys): run Chromium's journeys on css-lws and css-lws-only The Interop workflow tested the LWS servers with the pod tests alone, which sign in without a browser. A job per server now runs Chromium's user journeys there, with the app built with solid-memo-client, the client that exchanges the ID token for a storage's. It restores the images the pod job caches, and reports as the other Interop jobs do: it gates nothing. The workflow now also runs on changes to the app and to the journeys. Refs: #594 --- .github/workflows/interop.yml | 71 ++++++++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 1 deletion(-) diff --git a/.github/workflows/interop.yml b/.github/workflows/interop.yml index 4a00ecff9..74e2c97c7 100644 --- a/.github/workflows/interop.yml +++ b/.github/workflows/interop.yml @@ -6,7 +6,8 @@ name: Interop # The app's use cases and Solid adapters against the advisory Solid # servers (docs/testing.md), whose results are reported, not a gate: one -# job per server, as CI runs the blocking ones. A job fails only on a test +# job per server, as CI runs the blocking ones; and the user journeys in +# Chromium against the servers with Linked Web Storage. A job fails only on a test # failing that its servers//expected-failures.json does not list, or # when no test ran; the job's summary says what passed, what failed as # expected and why (by its cause on docs/interop.md), and what passes @@ -21,6 +22,8 @@ on: branches: [main] paths: - e2e/pod/** + - e2e/journeys/** + - apps/** - packages/** - ns/** - decks/** @@ -97,3 +100,69 @@ jobs: name: pod-logs-${{ matrix.server }} path: e2e/pod/logs/ retention-days: 7 + + # The user journeys in Chromium (docs/testing.md#user-journeys) against + # each server with Linked Web Storage, behind TLS as its signed-in + # instance, which the app signs in to with solid-memo-client: Inrupt's + # client gives no ID token to exchange for a storage's (docs/lws.md). + # The images come from the pod job's cache entries, which this job + # only restores. Any journey failing fails the job. Timing-dependent: + # end to end, on a shared runner. + journeys: + name: journeys (${{ matrix.server }}) + if: >- + !startsWith(github.event.head_commit.message, 'chore(release): ') + runs-on: ubuntu-latest + timeout-minutes: 30 + strategy: + fail-fast: false + matrix: + server: [css-lws, css-lws-only] + env: + JOURNEY_SERVER: ${{ matrix.server }} + SOLID_CLIENT: solid-memo-client + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version-file: .nvmrc + cache: npm + - run: npm ci --ignore-scripts + - id: key + run: echo "key=$(node e2e/pod/servers.ts key ${{ matrix.server }})" >> "$GITHUB_OUTPUT" + - uses: ./.github/actions/e2e-images + with: + name: pod-${{ matrix.server }} + key: ${{ steps.key.outputs.key }} + - run: npm run css:prepare -w @solid-memo/e2e-journeys + # The app alone, without the docs' diagrams, which need Chromium + # and have nothing of the client. + - run: npx turbo run build --filter=@solid-memo/web + env: + DOCS_DIAGRAMS: skip + - id: playwright + run: echo "version=$(node -p "require('@playwright/test/package.json').version")" >> "$GITHUB_OUTPUT" + - uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0 + id: browsers + with: + path: ~/.cache/ms-playwright + key: playwright-${{ runner.os }}-${{ steps.playwright.outputs.version }}-chromium + - if: steps.browsers.outputs.cache-hit != 'true' + run: npx playwright install chromium + - uses: ./.github/actions/playwright-deps + with: + browser: chromium + - run: npm run journeys -- --project=chromium + - if: always() + run: node e2e/journeys/harness/summary.ts journeys-${{ matrix.server }}-${{ github.run_attempt }} + - if: failure() + uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 + with: + name: journeys-${{ matrix.server }}-${{ github.run_attempt }} + path: | + e2e/journeys/playwright-report/ + e2e/journeys/test-results/ + e2e/journeys/logs/ + retention-days: 7 From 4621271b8dabdd3d27100e634013dbc7b4ceff30 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sun, 8 Nov 2026 03:18:08 +0100 Subject: [PATCH 17/19] docs(lws): leave the app's gaps to the interop report, and explain the keys The page listed, under "What LWS can't give the app yet", things the app doesn't use: live updates, which it never subscribes to, and a type index the server keeps, which no Solid server keeps either. Its first entry, "a server that takes LWS requests alone", read as if css-lws-only were such a server, though it takes the Solid Protocol's requests. What the tests find on a server with LWS is the interop report's to say, entry by entry as a test hits it, so the list goes and the page points there. The paragraph on the server's keys said what the harness does before why. It now gives the cause first: the fork makes its signing key on first use, and exchanges made at once on a new server each make one. Refs: #594 --- docs/docs.ttl | 2 +- docs/lws.md | 34 +++++++++++----------------------- e2e/pod/servers.ts | 2 +- 3 files changed, 13 insertions(+), 25 deletions(-) diff --git a/docs/docs.ttl b/docs/docs.ttl index a7fc7b96c..283f24f48 100644 --- a/docs/docs.ttl +++ b/docs/docs.ttl @@ -82,7 +82,7 @@ schema:isPartOf <#the-code> ; schema:position 46 ; schema:learningResourceType "explanation" ; - dcterms:abstract "how the app works on a Linked Web Storage server: finding the storage, why it needs the Solid Protocol's requests too, signing in with a token exchange, and what LWS can't give the app yet."@en . + dcterms:abstract "how the app works on a Linked Web Storage server: finding the storage, why it needs the Solid Protocol's requests too, signing in with a token exchange, and how it's tested."@en . a schema:TechArticle ; diff --git a/docs/lws.md b/docs/lws.md index 5b798015a..ceae88d22 100644 --- a/docs/lws.md +++ b/docs/lws.md @@ -4,8 +4,9 @@ Solid Memo is written for the Solid Protocol. The W3C's Linked Web Storage (LWS) Working Group is writing its successor, [LWS Protocol 1.0](https://www.w3.org/TR/lws10-core/), still a Working Draft. This page explains how the app works on a storage that speaks -LWS, why each part is made the way it is, and what LWS can't give it -yet. The servers it's tested on are in [testing.md](testing.md). +LWS, and why each part is made the way it is. The servers it's tested +on are in [testing.md](testing.md), and what the tests find on them in +[interop.md](interop.md). ## What differs @@ -92,24 +93,11 @@ logs the account in on the authorization-code flow and exchanges its ID token for the storage's token (`harness/podToken.ts`). The Solid-OIDC token that client credentials give is no LWS storage's. -**Why the harness reads the server's keys first.** On a newly started -server, token exchanges made side by side can each make the key the -server signs with. The server then refuses every token but those of the -key it kept ("signature verification failed"). One read of its keys -(`/.well-known/lws/jwks`) before the journeys start makes the one key. The fork's code is where this belongs fixed. - -## What LWS can't give the app yet - -- **A server that takes LWS requests alone.** The app needs the Solid - Protocol's requests ([why](#why-the-app-needs-solid-requests)). -- **A release anyone can read.** LWS defines no access control. The app - writes a release's ACL or ACR where the server has one; the Community - Solid Server with LWS keeps Web Access Control, so a release is made - public there. On a server with LWS alone, a release is published - readable by its owner alone, and asking to make it public fails - with `publicAccessRefused` - ([releasePublishing.ts](../packages/application/src/releasePublishing.ts)). -- **Live updates.** LWS has only webhooks, which a page in a browser - can't receive. The app doesn't use notifications. -- **A type index the server keeps.** The app keeps its own type index - documents, which an LWS server holds as any other data. +**Why the harness reads the server's keys first.** The fork makes the +key it signs tokens with the first time it needs one. On a newly started +server, several token exchanges at once can each make a key, and the +server keeps only one of them. A token signed with any other key is then +refused ("signature verification failed"). So the harness reads the +server's keys (`/.well-known/lws/jwks`) once before the journeys start. +That read makes the key, and every exchange after it uses the same one. +The fix belongs in the fork. diff --git a/e2e/pod/servers.ts b/e2e/pod/servers.ts index d0431446a..ed8046748 100644 --- a/e2e/pod/servers.ts +++ b/e2e/pod/servers.ts @@ -196,7 +196,7 @@ export const SERVERS = { podPath: "", readyPath: "", startTimeoutMs: 60_000, - // css-lws's image, run with LWS alone: what the app does where Solid's own requests are not taken (docs/lws.md). + // css-lws's image, with the fork's LWS-alone configuration, which still takes the Solid Protocol's requests (docs/lws.md). version: () => `${dockerfileArg("css-lws", "CSS_LWS_VERSION")} (${dockerfileArg("css-lws", "CSS_LWS_SHA").slice(0, 8)})`, signedIn: { login: "css-lws", host: "127.0.0.1" }, From 4cd9eaf387fa28c1de65641a248eafd0b4583bea Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sun, 8 Nov 2026 03:31:50 +0100 Subject: [PATCH 18/19] test(journeys): fail Interop's journeys only on failures their server doesn't expect Any journey failing on css-lws or css-lws-only failed its Interop job, so a cause found there could only be fixed or left red: the journeys had no list of expected failures, as the pod tests have. A failure of the app on a server with Linked Web Storage is what the job is there to report, not to hide. harness/compare.ts reads Playwright's JSON report and compares it with e2e/journeys/servers//expected-failures.json, each journey named by its file, its describe blocks and its title, and each failure by its cause on docs/interop.md, as e2e/pod/compare.ts does for the pod tests. The job fails on a journey failing that the list doesn't name, on an error outside any journey (a teardown that throws), which Playwright's exit code caught before, and when none ran. Its summary says what passes though listed, and the expected failures by cause. The traces of every failed journey are kept, expected or not. interop.test.ts keeps the journeys' lists in step with the page too, a server's causes in one section. Refs: #594 --- .github/workflows/interop.yml | 23 +++-- docs/interop.md | 11 +- docs/testing.md | 11 +- e2e/journeys/harness/compare.test.ts | 137 +++++++++++++++++++++++++ e2e/journeys/harness/compare.ts | 147 +++++++++++++++++++++++++++ e2e/pod/interop.test.ts | 28 +++-- 6 files changed, 333 insertions(+), 24 deletions(-) create mode 100644 e2e/journeys/harness/compare.test.ts create mode 100644 e2e/journeys/harness/compare.ts diff --git a/.github/workflows/interop.yml b/.github/workflows/interop.yml index 74e2c97c7..b037885b2 100644 --- a/.github/workflows/interop.yml +++ b/.github/workflows/interop.yml @@ -8,10 +8,11 @@ name: Interop # servers (docs/testing.md), whose results are reported, not a gate: one # job per server, as CI runs the blocking ones; and the user journeys in # Chromium against the servers with Linked Web Storage. A job fails only on a test -# failing that its servers//expected-failures.json does not list, or -# when no test ran; the job's summary says what passed, what failed as -# expected and why (by its cause on docs/interop.md), and what passes -# though listed. A release commit, +# or journey failing that its server's expected-failures.json does not +# list (e2e/pod/servers// for the tests, e2e/journeys/servers// +# for the journeys), or when none ran; the job's summary says what +# passed, what failed as expected and why (by its cause on +# docs/interop.md), and what passes though listed. A release commit, # whose push its paths match (package.json, package-lock.json), runs # nothing: it changes only the version's files, so its code is the # commit's before it, which Interop already ran on @@ -106,8 +107,9 @@ jobs: # instance, which the app signs in to with solid-memo-client: Inrupt's # client gives no ID token to exchange for a storage's (docs/lws.md). # The images come from the pod job's cache entries, which this job - # only restores. Any journey failing fails the job. Timing-dependent: - # end to end, on a shared runner. + # only restores. compare.ts fails the job on a journey failing that the + # server's e2e/journeys/servers//expected-failures.json does not + # list. Timing-dependent: end to end, on a shared runner. journeys: name: journeys (${{ matrix.server }}) if: >- @@ -154,10 +156,15 @@ jobs: - uses: ./.github/actions/playwright-deps with: browser: chromium - - run: npm run journeys -- --project=chromium + # compare.ts says whether the run went as expected, not Playwright. + - id: journeys + run: npm run journeys -- --project=chromium + continue-on-error: true - if: always() run: node e2e/journeys/harness/summary.ts journeys-${{ matrix.server }}-${{ github.run_attempt }} - - if: failure() + - run: node e2e/journeys/harness/compare.ts ${{ matrix.server }} + # Every failed journey's trace, expected or not. + - if: failure() || steps.journeys.outcome == 'failure' uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2 with: name: journeys-${{ matrix.server }}-${{ github.run_attempt }} diff --git a/docs/interop.md b/docs/interop.md index 26bd63e2a..ffb4cfd71 100644 --- a/docs/interop.md +++ b/docs/interop.md @@ -8,13 +8,15 @@ it, with no Solid Memo involved. The tests and how they run are in [testing.md](testing.md). The Interop workflow runs them weekly and on changes to the app's Solid -code. Each server's `e2e/pod/servers//expected-failures.json` names -the tests a cause fails, by the cause's id on this page. +code, and the user journeys on the servers with Linked Web Storage. +Each server's `e2e/pod/servers//expected-failures.json` names the +tests a cause fails, and its `e2e/journeys/servers//expected-failures.json` +the journeys, by the cause's id on this page. A cause here is one that the tests hit and a live run reproduced. The servers are set up for testing, and each entry says when the setup plays a part. A server not on this page, or with no entries, passes -every test. +every test and journey. ## How to read an entry @@ -26,7 +28,8 @@ every test. - **Reproduction** uses `$POD` for the pod's root URL, ending in `/`. The test pods are open to anyone, so the requests carry no credentials, unless an entry says it's from the - [signed-in tests](testing.md#the-signed-in-tests). Headers and bodies are trimmed to what matters. + [signed-in tests](testing.md#the-signed-in-tests) or the + [user journeys](testing.md#user-journeys), which sign in. Headers and bodies are trimmed to what matters. - **Upstream** links the issue or pull request that tracks the cause, or says that none does yet. diff --git a/docs/testing.md b/docs/testing.md index 69b4f1678..7067f411d 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -446,7 +446,16 @@ their signed-in instances behind TLS from the image e2e/pod builds. The app signs in to them only when built with `SOLID_CLIENT=solid-memo-client` ([lws.md](lws.md#how-its-tested)). The Interop workflow runs Chromium's journeys on both and reports them, -as it does the pod tests: they gate nothing. +as it does the pod tests: they gate nothing. A job fails only when no +journey ran, or a journey fails that +`e2e/journeys/servers//expected-failures.json` +(`{ " > ": "" }`, the journey named by its file, +its describe blocks and its title) doesn't list. Each cause is an entry +of [interop.md](interop.md), in the same server's section as its pod +tests' causes, and `e2e/pod/interop.test.ts` keeps the lists and the +page in step. [harness/compare.ts](../e2e/journeys/harness/compare.ts) +names these in the job's summary, with what passes though listed and +the expected failures by cause. Further journeys go through: diff --git a/e2e/journeys/harness/compare.test.ts b/e2e/journeys/harness/compare.test.ts new file mode 100644 index 000000000..85c55c41e --- /dev/null +++ b/e2e/journeys/harness/compare.test.ts @@ -0,0 +1,137 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { describe, expect, it } from "vitest"; +import { compare, pageOf, type Results, summary } from "./compare.ts"; + +const results: Results = { + suites: [ + { + title: "smoke.journey.ts", + specs: [ + { title: "logs in", file: "smoke.journey.ts", tests: [{ status: "expected" }] }, + { title: "studies", file: "smoke.journey.ts", tests: [{ status: "unexpected" }] }, + { title: "imports", file: "smoke.journey.ts", tests: [{ status: "unexpected" }] }, + { title: "waits", file: "smoke.journey.ts", tests: [{ status: "skipped" }] }, + ], + suites: [ + { + title: "a course", + specs: [ + { title: "plays", file: "smoke.journey.ts", tests: [{ status: "flaky" }] }, + { title: "publishes", file: "smoke.journey.ts", tests: [{ status: "expected" }] }, + ], + }, + ], + }, + ], +}; + +describe("compare", () => { + it("names each journey by its file, its describe blocks and its title, and sorts it by what it did", () => { + const expected = { + "smoke.journey.ts > studies": "why", + "smoke.journey.ts > a course > publishes": "why", + }; + expect(compare(results, expected)).toEqual({ + passed: ["smoke.journey.ts > logs in", "smoke.journey.ts > a course > plays"], + skipped: ["smoke.journey.ts > waits"], + known: ["smoke.journey.ts > studies"], + failed: ["smoke.journey.ts > imports"], + fixed: ["smoke.journey.ts > a course > publishes"], + }); + }); + + it("fails a run with an error outside any journey, as its list may expect", () => { + const run = { ...results, errors: [{ message: "teardown threw" }] }; + expect(compare(run, {}).failed).toEqual([ + "smoke.journey.ts > studies", + "smoke.journey.ts > imports", + "(outside any journey)", + ]); + expect(compare(run, { "(outside any journey)": "why" }).known).toEqual([ + "(outside any journey)", + ]); + }); + + it("finds no journeys in a report without suites", () => { + expect(compare({}, {})).toEqual({ passed: [], skipped: [], known: [], failed: [], fixed: [] }); + }); +}); + +describe("summary", () => { + it("says what failed anew, what passes though listed, and each expected failure under its cause", () => { + const expected = { + "a.journey.ts > one": "lws-x", + "a.journey.ts > two": "lws-x", + "a.journey.ts > three": "lws-y", + "a.journey.ts > four": "lws-y", + }; + const text = summary( + "css-lws-only", + { + passed: ["a.journey.ts > five"], + skipped: [], + known: ["a.journey.ts > one", "a.journey.ts > two", "a.journey.ts > three"], + failed: ["a.journey.ts > six"], + fixed: ["a.journey.ts > four"], + }, + expected, + "docs/interop.md", + ); + expect(text).toBe( + [ + "### Journeys on css-lws-only: new failures", + "", + "1 passed, 0 skipped, 3 failed as expected, 1 failed anew, 1 passed though listed.", + "", + "**Failed anew**", + "", + "- a.journey.ts > six", + "", + "**Passed though listed (take them off expected-failures.json)**", + "", + "- a.journey.ts > four", + "", + "**Failed as expected, by cause ([interop.md](docs/interop.md))**", + "", + "- [lws-x](docs/interop.md#lws-x): 2 journeys", + " - a.journey.ts > one", + " - a.journey.ts > two", + "- [lws-y](docs/interop.md#lws-y): 1 journey", + " - a.journey.ts > three", + "", + ].join("\n"), + ); + }); + + it("says a run with nothing unexpected went as expected", () => { + expect( + summary( + "css-lws", + { passed: ["a.journey.ts > one"], skipped: [], known: [], failed: [], fixed: [] }, + {}, + "docs/interop.md", + ), + ).toBe( + "### Journeys on css-lws: as expected\n\n1 passed, 0 skipped, 0 failed as expected, 0 failed anew, 0 passed though listed.\n", + ); + }); +}); + +describe("pageOf", () => { + it("links the page at the run's commit on CI", () => { + expect( + pageOf({ + GITHUB_SERVER_URL: "https://github.com", + GITHUB_REPOSITORY: "solid-memo/website", + GITHUB_SHA: "abc", + }), + ).toBe("https://github.com/solid-memo/website/blob/abc/docs/interop.md"); + }); + + it("names the page's path elsewhere", () => { + expect(pageOf({})).toBe("docs/interop.md"); + }); +}); diff --git a/e2e/journeys/harness/compare.ts b/e2e/journeys/harness/compare.ts new file mode 100644 index 000000000..0bfeba9f7 --- /dev/null +++ b/e2e/journeys/harness/compare.ts @@ -0,0 +1,147 @@ +// SPDX-FileCopyrightText: 2026 Anton Wiklund and Solid Memo contributors +// +// SPDX-License-Identifier: MIT + +import { appendFileSync, existsSync, readFileSync } from "node:fs"; +import { join } from "node:path"; + +/** What Playwright's JSON reporter writes, as far as it is read here. */ +export interface Results { + suites?: Suite[]; + /** What failed outside any journey: a global teardown, a worker after its journeys. */ + errors?: unknown[]; +} +interface Suite { + title: string; + specs?: { title: string; file: string; tests: { status: string }[] }[]; + suites?: Suite[]; +} + +export interface Comparison { + passed: string[]; + skipped: string[]; + /** Failed, as its server's expected-failures.json says it does. */ + known: string[]; + /** Failed, and not listed: what makes the run fail. */ + failed: string[]; + /** Listed, and passed: to take off the list. */ + fixed: string[]; +} + +/** + * A journey's name: its file, then the titles of the describe blocks it's + * in, then its own. A file's suite is titled by the file, so the first + * title is left out. + */ +function keysOf(suite: Suite, describes: string[]): { key: string; status: string }[] { + return [ + ...(suite.specs ?? []).flatMap((spec) => + spec.tests.map((test) => ({ + key: [spec.file, ...describes, spec.title].join(" > "), + status: test.status, + })), + ), + ...(suite.suites ?? []).flatMap((inner) => keysOf(inner, [...describes, inner.title])), + ]; +} + +/** + * Each journey of a run, by what it did and whether its server's list + * expects it to fail. Playwright says "unexpected" of a journey that + * failed, "expected" of one that passed and "flaky" of one that passed + * on a retry. + */ +export function compare(results: Results, expected: Record): Comparison { + const comparison: Comparison = { passed: [], skipped: [], known: [], failed: [], fixed: [] }; + for (const { key, status } of (results.suites ?? []).flatMap((file) => keysOf(file, []))) { + if (status === "unexpected") comparison[key in expected ? "known" : "failed"].push(key); + else if (status === "skipped") comparison.skipped.push(key); + else comparison[key in expected ? "fixed" : "passed"].push(key); + } + // An error outside any journey fails the run as Playwright's exit code + // did, with every journey passed: that is a failure too. + if ((results.errors ?? []).length > 0) { + const key = "(outside any journey)"; + comparison[key in expected ? "known" : "failed"].push(key); + } + return comparison; +} + +/** + * The comparison as the job's summary says it, each expected failure + * under its cause: its id on `page` (docs/interop.md), where the cause is + * described for the server's developers. + */ +export function summary( + id: string, + comparison: Comparison, + expected: Record, + page: string, +): string { + const { passed, skipped, known, failed, fixed } = comparison; + const lines = [ + `### Journeys on ${id}: ${failed.length > 0 ? "new failures" : "as expected"}`, + "", + `${passed.length} passed, ${skipped.length} skipped, ${known.length} failed as expected, ${failed.length} failed anew, ${fixed.length} passed though listed.`, + ]; + const list = (title: string, keys: string[]) => + keys.length > 0 && lines.push("", `**${title}**`, "", ...keys.map((key) => `- ${key}`)); + list("Failed anew", failed); + list("Passed though listed (take them off expected-failures.json)", fixed); + const byCause = new Map(); + for (const key of known) + byCause.set(expected[key]!, [...(byCause.get(expected[key]!) ?? []), key]); + if (byCause.size > 0) { + lines.push("", `**Failed as expected, by cause ([interop.md](${page}))**`, ""); + for (const [cause, keys] of byCause) + lines.push( + `- [${cause}](${page}#${cause}): ${keys.length} ${keys.length === 1 ? "journey" : "journeys"}`, + ...keys.map((key) => ` - ${key}`), + ); + } + return `${lines.join("\n")}\n`; +} + +/** docs/interop.md at the run's commit on CI, and its path elsewhere. */ +export const pageOf = (env: Record) => + env.GITHUB_SERVER_URL && env.GITHUB_REPOSITORY && env.GITHUB_SHA + ? `${env.GITHUB_SERVER_URL}/${env.GITHUB_REPOSITORY}/blob/${env.GITHUB_SHA}/docs/interop.md` + : "docs/interop.md"; + +/** + * `node harness/compare.ts `: whether the journeys' run on an + * advisory server went as expected (docs/testing.md#user-journeys), said + * in the job's summary on CI. It reads test-results/results.json, and + * fails on a journey failing that servers//expected-failures.json + * ({ " > ": "" }, the cause's id on + * docs/interop.md) does not list, and when no journey ran. + */ +if (import.meta.main) { + const [id] = process.argv.slice(2); + if (id === undefined) throw new Error("Usage: node harness/compare.ts "); + const root = join(import.meta.dirname, ".."); + const listFile = join(root, "servers", id, "expected-failures.json"); + const expected = existsSync(listFile) + ? (JSON.parse(readFileSync(listFile, "utf8")) as Record) + : {}; + const resultFile = join(root, "test-results", "results.json"); + const results = existsSync(resultFile) + ? (JSON.parse(readFileSync(resultFile, "utf8")) as Results) + : undefined; + const comparison = results && compare(results, expected); + const ran = + comparison !== undefined && + comparison.passed.length + + comparison.known.length + + comparison.failed.length + + comparison.fixed.length > + 0; + const text = ran + ? summary(id, comparison, expected, pageOf(process.env)) + : `### Journeys on ${id}: no results\n\nNo journey ran: the server or the app did not start (the job's log says).\n`; + if (process.env.GITHUB_STEP_SUMMARY) appendFileSync(process.env.GITHUB_STEP_SUMMARY, text); + console.log(text); + for (const key of comparison?.fixed ?? []) + console.log(`::warning::${id}: "${key}" passes; take it off expected-failures.json`); + if (!ran || comparison.failed.length > 0) process.exitCode = 1; +} diff --git a/e2e/pod/interop.test.ts b/e2e/pod/interop.test.ts index 16e3d9350..4b69f43f0 100644 --- a/e2e/pod/interop.test.ts +++ b/e2e/pod/interop.test.ts @@ -23,21 +23,27 @@ const sections = page })) .filter((section) => section.headings.length > 0); +/** The causes a server's list in `dir` names, none where it has no list. */ +const causesOf = (dir: string, id: string) => { + const file = join(dir, id, "expected-failures.json"); + return existsSync(file) + ? Object.values(JSON.parse(readFileSync(file, "utf8")) as Record) + : []; +}; +// The pod tests' lists, and the journeys' (e2e/journeys/servers//). const servers = join(import.meta.dirname, "servers"); -const lists = readdirSync(servers) - .filter((id) => existsSync(join(servers, id, "expected-failures.json"))) +const journeyServers = join(import.meta.dirname, "../journeys/servers"); +const lists = [ + ...new Set([ + ...readdirSync(servers), + ...(existsSync(journeyServers) ? readdirSync(journeyServers) : []), + ]), +] .map((id) => ({ id, - causes: new Set( - Object.values( - JSON.parse(readFileSync(join(servers, id, "expected-failures.json"), "utf8")) as Record< - string, - string - >, - ), - ), + causes: new Set([...causesOf(servers, id), ...causesOf(journeyServers, id)]), })) - // A server with an empty list fails nothing, and the page leaves it out. + // A server whose lists are empty fails nothing, and the page lists no cause of it. .filter((list) => list.causes.size > 0); describe("docs/interop.md", () => { From e88a3c183b89dd7724db441ed0c22356538d55c6 Mon Sep 17 00:00:00 2001 From: Anton Wiklund Date: Sun, 8 Nov 2026 03:31:50 +0100 Subject: [PATCH 19/19] docs(interop): report on the servers with Linked Web Storage css-lws and css-lws-only run the pod tests and the user journeys in the Interop workflow, but the interop report named neither, so a cause found there had no section to go in. Each now has one: the fork's commit and configuration, how the tests run it, and that every test and journey passes. css-lws-only takes the Solid Protocol's requests too, which the section says, so its results aren't read as the app on LWS requests alone. An entry may give a clause of LWS Protocol 1.0 as its requirement, or say that LWS doesn't define what the app needs. Refs: #594 --- docs/interop.md | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/docs/interop.md b/docs/interop.md index ffb4cfd71..7a6ef9427 100644 --- a/docs/interop.md +++ b/docs/interop.md @@ -23,8 +23,11 @@ every test and journey. - **Requirement** is the strongest clause the behavior goes against, with its level: a MUST of the [Solid Protocol 0.11](https://solidproject.org/TR/protocol) or of [Web Access Control 1.0.0](https://solidproject.org/TR/wac), a - definition in an RDF, SPARQL or HTTP specification, or none. An - optional feature done wrong says so. + definition in an RDF, SPARQL or HTTP specification, or none. On a + server with Linked Web Storage, it may be a clause of + [LWS Protocol 1.0](https://www.w3.org/TR/lws10-core/). An optional + feature done wrong says so. So does something the app needs that LWS + doesn't define. - **Reproduction** uses `$POD` for the pod's root URL, ending in `/`. The test pods are open to anyone, so the requests carry no credentials, unless an entry says it's from the @@ -546,3 +549,28 @@ The same PUT to `card.ttl` is served as sent. **Reported to the maintainers in private.** It gets an entry like the others once a fixed release is out. + +## Community Solid Server with Linked Web Storage + +`css-lws`: the Linked Web Storage branch of a fork of the Community +Solid Server, not yet merged +([jeswr/CommunitySolidServer#126](https://github.com/jeswr/CommunitySolidServer/pull/126)), +at commit +[`ef6d664f`](https://github.com/jeswr/CommunitySolidServer/tree/ef6d664f1253662cbfcb0f1786fe45d32849e067) +(version 8.0.0-alpha.3). Its configuration is the fork's +`config/solid-lws.json`: the Solid Protocol and LWS side by side, with +Web Access Control. The pod tests run it in memory, its root open to +anyone. The signed-in tests and the user journeys run its signed-in +instance, behind TLS. + +No causes: every pod test and every journey passes. + +## Community Solid Server with Linked Web Storage alone + +`css-lws-only`: the same commit, with the fork's `config/lws.json`, LWS +alone with Web Access Control. This configuration takes the Solid +Protocol's requests too, so the app's requests are answered as on +`css-lws`. The pod tests and the signed-in instance are run as for +`css-lws`. + +No causes: every pod test and every journey passes.