Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ jobs:
- run: npm ci --no-audit --no-fund
working-directory: frontend
- run: node --disable-warning=MODULE_TYPELESS_PACKAGE_JSON tests/integration/postgres_recovery.mjs
- run: node --disable-warning=MODULE_TYPELESS_PACKAGE_JSON tests/integration/postgres_quota.mjs
edge:
runs-on: ubuntu-latest
timeout-minutes: 5
Expand Down
43 changes: 43 additions & 0 deletions docs/quota/00_AUDIT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Guest quota audit

11 November 2026. Local baseline: `fb38df3`. Source inspection only: production schema, dashboards and deployment are not verified. The original Downloads prompt is no longer present; MASTER_PROMPT.md records the corrected implementation contract from the reviewed text.

## Acceptance graph

```mermaid
flowchart TD
W[Web batch / extension batch] --> I[Server-issued guest cookie]
I --> R[Atomic batch quota + storage + job reservation]
R --> S[Per-file signed upload issuance]
S --> U[Private Storage upload]
U --> J[start_uploaded_job / enqueue root]
J --> C[claim_work_items]
C --> Q[Database quota start fence]
Q --> B[N / O / Z backend]
B --> F[Existing finalization and output signing]
F --> D[Result access independent of remaining quota]
```

Existing entry points: `frontend/app/api/uploads/create/route.js`, `api/convert`, `api/jobs/[id]/start`, `advance`, `download`, authenticated tick and admin routes. `jobService.js` calls `create_reserved_upload_job`; that transaction already enforces storage forecasts and the 32-active-root ceiling. `publicAdmission.js` adds global and keyed-IP request budgets, not a daily file quota. `supabaseRest.js` uses service-role PostgREST, not a direct SQL connection. Workers require the shared backend secret. Anonymous REST/RPC privileges remain revoked.

The extension already uploads files; its transport previously omitted credentials. Web and extension must bootstrap the same host cookie, send credentials only to the canonical app, and reserve full selections before upload. An extension update and native browser acceptance are required before enabling enforcement. Old clients receive a structured update-required error, never a bypass.

## Decisions

- DB policy is the enforcement switch, default disabled. All server entry points consult it; unknown policy fails closed. Schema must be applied before frontend deployment.
- Guest 20, rolling 24 hours from admission. A reserved file occupies a slot immediately. First durable worker claim commits `started`; completed/failed processing never refunds. Pending slots expire after three hours; expired slots cannot claim workers. No worker page/ZIP child charges.
- Entire batch reserves atomically, including existing storage/active-job checks. Insufficient quota rejects all files. User removes files and resubmits; no partial implicit conversion.
- Manifest-bound UUID idempotency key returns original jobs. Different manifest with same key is a conflict. Storage signing/HTTP uncertainty retains reservations for safe replay.
- Upload signing is restricted to 45 minutes after admission, preserving the existing three-hour storage guard for two-hour signed URLs. Replays can resume an existing upload without issuing another upload URL.
- Protected ledger has no cascading job FK: deleting results must not refund charged usage. Existing UUID-capability result retrieval is preserved. Quota identity does not silently become document ownership or account membership.
- A separate server secret signs guest cookies and hashes tokens. Cookie clearing creates a new identity; global abuse controls remain. No person-level guarantee.
- Same-subject locking serializes reservations, releases and first worker charge. Read database clock after the lock. No raw cookies, file contents or signed URLs in logs.
- Direct multipart processing is disabled with a structured use-storage response while quota enforcement is enabled. Requests already carrying body bytes cannot be retroactively stopped at the network edge.
- Shared policy activation must not grandfather newly created unmetered jobs. Only jobs predating initial activation retain legacy processing access.
- Account promotion, passwords and SMTP remain design-only. Future usage merge must preserve distinct events and cannot grant result access on a shared browser.

## Risks and evidence gates

Use real PostgreSQL multi-connection concurrency tests in addition to PGlite. Test fresh-cookie multi-tab bootstrap, extension cookie behavior, blocked cookies, signed-URL replay after reservation expiry, processing/cleanup races, lost HTTP responses, differing-manifest replay, existing result retrieval and all six UI languages. Keep feature disabled until native extension and production migration checks pass. No production load tests or account/email activation are authorized by this rollout.

48-hour default document retention and KEEP/EXTEND remain intact. Existing storage admission keeps the three-hour guard independently of quota release. Ledger expiry is not permission to delete Storage objects or alter retained results.
17 changes: 17 additions & 0 deletions docs/quota/FUTURE_AUTH_RESEND.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Verified accounts and transactional email

Design only, 12 November 2026. The guest release uses a secure browser cookie. The account milestone adds Supabase Auth email/password and confirmed email, with Resend custom SMTP for transactional mail.

## Account quota

- Validate the Supabase session server-side and use the verified user UUID. Unverified sessions retain the guest limit. Keep credentials and service keys server-only.
- The verified limit is 50 total top-level uploads over the same rolling 24 hours. Account activity is shared across devices.
- During sign-in or verification, lock the guest and account quota subjects in deterministic order. Associate that guest once and preserve unique event IDs, timestamps, in-flight reservations and prior account usage. Treat repeated association as idempotent. If the combined usage exceeds 50, reject new reservations until the required slots return.
- Decide how logout, browser sharing, account deletion and expired sessions affect guest association before implementation. Avoid granting result access as a side effect of reconciling quota usage. Keep upload/result access governed by their own authorization.

## Email setup

- Use the [Supabase email/password flow](https://supabase.com/docs/guides/auth/passwords) with confirmation and recovery. Allow only intended HTTPS redirects. Test expired, reused and invalid links and use neutral responses to reduce account enumeration.
- Verify a sending subdomain the operator controls, such as `auth.llmiser.devbehindyou.com` if selected. Add the records shown for that domain by [Resend domain verification](https://resend.com/docs/dashboard/domains/introduction). Review sender identity, authentication and DMARC before sending.
- Use the supported [Resend SMTP integration](https://resend.com/docs/send-with-supabase-smtp). Store the API key in the provider configuration. Verify current SMTP settings, [Resend pricing and allowances](https://resend.com/pricing) and [Supabase sending limits](https://supabase.com/docs/guides/auth/auth-smtp) at rollout. Keep file quota and email rate limits independent.
- Test verification, resend, reset, delivery failures and throttling with test addresses. Review abuse controls, accessibility, all supported app languages, shared-browser access, data retention and account deletion before exposing signup.
14 changes: 14 additions & 0 deletions docs/quota/IMPLEMENTATION_STATUS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Guest quota implementation status

12 November 2026. The branch contains the additive migration, atomic whole-batch admission, signed-upload and claim checks, quota UI and updated Chrome extension transport. It also includes the revised contract and future account design.

| Stage | Evidence |
| --- | --- |
| Local implementation | Database policy defaults disabled. Guest limit is 20 top-level files per rolling 24 hours. |
| Local extension checks | 31 tests passed. The extension built and passed package checks. |
| Database concurrency | Multi-connection PostgreSQL CI verified parallel batches, last-slot races, idempotent replay and claim-versus-release. |
| Native Chromium | A disposable browser profile with a native extension context and loopback fixtures verified shared guest identity in both bootstrap orders, missing-cookie rejection, batch rejection, the 21st-file denial, idempotent upload replay and download after exhaustion. A 20-file batch initially exposed a short-term request throttle. After the retry fix, all 20 files completed and the quota showed 20 used and zero reserved. |
| Hosted Storage | Staging acceptance still required with hosted Supabase Storage and the target Chrome configuration. The browser fixture used the app handlers and SQL migrations with simulated Storage signing and conversion workers. |
| Production | Migration, deployment and policy activation pending. |

The next release step is staging validation with the migration installed and the quota-aware web app and extension pointed at the same test environment. Follow [the rollout runbook](ROLLOUT.md). Account authentication and Resend are a [separate design milestone](FUTURE_AUTH_RESEND.md).
19 changes: 19 additions & 0 deletions docs/quota/MASTER_PROMPT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Revised implementation contract

11 November 2026. Supersedes ambiguous clauses in the reviewed LLMiser Quota/Limits/Auth/Resend prompt. Implement guest quota now; design authentication/Resend separately. Follow 00_AUDIT.md and the implementation status rather than historical MDify paths.

1. Preserve existing conversion, deadlines, queue, storage admission, retention and result retrieval. Work only in LLMiser; use current source and docs/ai-handover/HANDOVER.md and PROGRESS.md. No STATE.yaml is assumed.
2. Exactly 20 accepted top-level files per server-issued guest identity per rolling 24 hours. Reserve before signing uploads. Count pending and started work, never child tasks. Whole batches are atomic and rejected if they do not fit.
3. Idempotency binds subject + request UUID + complete normalized file manifest. Identical replay returns the same jobs; conflicting replay returns 409. Network uncertainty is not proof of failure and cannot trigger a refund.
4. First durable worker claim starts charging permanently for the admission window. Pending expiry/release fences future claims. Started failures remain charged. New submissions are new charges; recovery of an existing job is not.
5. Preserve independent storage authorization guards. Expired quota cannot authorize worker dispatch even if an old signed URL still uploads bytes. Do not shorten byte reservations when releasing file quota.
6. Expose only service-role RPC wrappers, preferably SECURITY INVOKER with explicit table privileges. Sensitive tables can remain private. Do not attempt to call a non-exposed schema directly through the current PostgREST client.
7. Extension backend uploads are a current required path. Bootstrap server identity and explicitly test cookie sharing with host permissions. Never send app cookies or secrets to Storage. Missing/blocked identity fails with actionable UI, not a new identity per file.
8. Keep result access separate from quota enforcement and preserve old recovery links. Later account quota merges must not imply document transfer.
9. Use structured errors: DAILY_FILE_LIMIT with actual remaining, requested count, next-slot time and request-fit time; capacity errors remain distinct. Permanent quota errors are not automatic processing retries. Refresh status on focus and after outcomes.
10. Guest UI uses all six app languages, accessible announcements and explicit rolling-window text. Near-limit <=5, pending shown separately, downloads never disabled for exhaustion.
11. DB flag defaults disabled; apply additive schema before code and enable only after CI, real PostgreSQL races, native extension/browser acceptance and production configuration checks. Fail closed on unknown policy. No automatic production schema replay.
12. Keep bounded cleanup and metadata-only diagnostics. Never log tokens, signed URLs or input contents. Distinguish implemented, tested, merged, deployed, enabled and production-verified.
13. Future-only: verified Supabase account gets 50 total, not 20+50. Usage reconciliation uses immutable event IDs and one-time guest association. Supabase Auth handles passwords; Resend custom SMTP is separate. No signup, password reset, DNS changes or email credentials in this guest release.

Deliver audit, additive SQL, server and client integration, regression/concurrency tests, operational runbook, future-auth design and honest milestone evidence. Existing broad PR/release authorization does not replace required validation.
22 changes: 22 additions & 0 deletions docs/quota/ROLLOUT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Guest file quota rollout

The additive migration ships with enforcement disabled. It extends the existing job and Storage path. The guest allowance is per browser identity and uses rolling admission timestamps. Global admission and storage capacity remain independent.

1. Run the branch checks, including `tests/integration/postgres_quota.mjs` against its disposable multi-connection PostgreSQL fixture. Inspect the exact tested branch revision.
2. Apply only `supabase/migrations/20261111000000_llmiser_guest_quota.sql` to an isolated Supabase project with the earlier migrations installed. Verify `select public.quota_policy();` shows disabled. Test the prior frontend against the additive schema. Avoid replaying the consolidated setup script against an existing deployment.
3. Configure `QUOTA_GUEST_SECRET` as a stable, independently generated random secret of at least 32 characters. Use one value per environment across instances. Rotation invalidates old guest identities and should be planned. The server refuses guest issuance if the secret is missing.
4. In staging, deploy the quota-aware app and build. Set `quota_private.policy.enabled` to true for its singleton row. Verify a real signed upload for each pool, 20 admissions followed by a rejected 21st, an entire batch rejection, resuming a successfully uploaded file, and an existing download after exhaustion. Check local-time display and keyboard operation.
5. Load the extension build in Chrome against an approved test setup. Verify the browser can share the host cookie between the app and the extension, including the browser's cookie restrictions. Use test fixtures. Verify that Storage requests omit credentials. The shipped extension targets the canonical production app and Storage origins, so use an isolated test build configured for staging for pre-release smoke tests.
6. Apply the same single migration in production before deploying this frontend. Keep policy disabled while verifying legacy uploads, recovery ticks and the new version. Make the updated extension available before enabling. Older clients receive an update-required response under enforcement.
7. Enable after the release evidence is recorded. Observe quota rejection counts, reservation expiry, orphan handling, queue delay and backend capacity through metadata-only diagnostics. Avoid logging cookies, upload URLs, filenames or file content.

Use this operator query in the target SQL Editor after independently verifying the project and environment:

```sql
select public.quota_policy();
update quota_private.policy set enabled = true where singleton;
```

Set `enabled = false` to suspend enforcement and allow ordinary admission again. Leave the additive migration installed so outstanding reserved jobs and new frontend calls retain their contracts. Existing metered jobs still obey their own reservation state. Before re-enabling, settle unmetered jobs created during the disabled period. Do not delete ledger rows, release started slots or reset the activation timestamp as a rollback procedure.

Signing closes after 45 minutes to leave headroom within the existing three-hour Storage guard. An existing input can still resume before its reservation expires. Unstarted quota slots release after expiry or terminal pre-start recovery. Started events remain charged for the original rolling window. The tick prunes eligible terminal ledger batches after seven days. This does not alter the 48-hour document retention policy.
6 changes: 4 additions & 2 deletions extension/PRIVACY.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
# LLMiser Chrome extension privacy

Version 0.2.0 — 24 October 2026. Operator: DevBehindYou.
Quota-aware revision — 11 November 2026. Operator: DevBehindYou.

The extension has one file Converter screen in a toolbar popup. It does not capture webpages or selected text, collect analytics, monitor browsing history or send chat messages. Page extraction and its libraries were removed in version 0.2.0.

Selecting or dropping files does not upload them. Clicking Convert files requests optional access to exactly `https://llmiser.devbehindyou.com` and `https://ppmbqgecdbrezxedyeur.supabase.co`. If granted, file metadata (filename, size, content type and profile) goes to LLMiser; bytes go directly to its private Supabase Storage using a signed URL. LLMiser's existing backends process those files and Markdown is downloaded to the popup. Requests omit cookies, reject redirects and validate signed storage links against the expected project/bucket/job. No backend secret or service-role key ships in the extension. Normal service/network logs, processing and the default 48-hour retention with documented administrative exceptions apply. See https://llmiser.devbehindyou.com/privacy.
Selecting or dropping files does not upload them. Clicking Convert files requests optional access to exactly `https://llmiser.devbehindyou.com` and `https://ppmbqgecdbrezxedyeur.supabase.co`. If granted, file metadata (filename, size, content type and profile) goes to LLMiser; bytes go directly to its private Supabase Storage using a signed URL. LLMiser's existing backends process those files and Markdown is downloaded to the popup. Requests to LLMiser include its service cookie for the guest file allowance; Storage requests omit cookies. Redirects are rejected and signed storage links are validated against the expected project/bucket/job. No backend secret or service-role key ships in the extension. Normal service/network logs, processing and the default 48-hour retention with documented administrative exceptions apply. See https://llmiser.devbehindyou.com/privacy.

When file limits are enabled, LLMiser issues a secure, HttpOnly guest cookie with a 90-day lifetime. The server stores a keyed hash, request/file identifiers, a manifest hash and quota timestamps, not the raw cookie. Quota metadata is eligible for bounded cleanup after seven days; records for active jobs remain until recovery settles. The whole selected batch is reserved before uploading. Started conversion counts even if it fails; retries of the same admitted job do not count again. Unstarted reservations expire after three hours. Existing output remains accessible when the allowance is exhausted. Cookie access depends on browser settings; blocked identity access prevents new conversion rather than bypassing limits.

The `storage` permission keeps one editable file draft and up to 20 recovery records in `chrome.storage.session`. A draft contains Markdown, filename, capture time and warnings. Recovery records contain only job ID, filename and upload/processing phase; no signed URL, access token, original file bytes or browsing URL is retained in these records. Job IDs allow retrieval of those jobs; keep your browser session private. Chrome session storage is cleared when Chrome closes or the extension is disabled, reloaded or updated, and Chrome may clear it independently. Old page-capture drafts are not restored in this file-only version.

Expand Down
Loading
Loading