Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .forge/features/career-job-storage/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Career Job Storage

Implement storage for #647, part of #633. Members will explore jobs, track
applications, find others in process, and configure alerts. Officers will register
employer boards for supported providers. This slice adds storage and tests only.

Agreed behavior: seven new tables; one primary Discord post total per job; multiple
locations per listing; shared daily digests with separate track opt-ins. Retain
inactive jobs for application history. Rolling-feed disappearance or failed polls
cannot establish closure. Explicit employer evidence determines track; otherwise
keep the first accepted track. Reclassification never authorizes another post.

Reuse existing company identity without auto-approving scraped companies. Members
control application visibility and private notes always remain private. No UI,
API workflows, production migration, old preference import or historical job
import is included. Acceptance: isolated migration succeeds, existing data survives,
opt-ins default off, constraints reject invalid/duplicate records, consumers compile.
101 changes: 101 additions & 0 deletions .forge/features/career-job-storage/srd.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Career Job Storage SRD

Status: Implementing the design agreed in the local November 3 conversation. The
#633 GitHub design review is still outstanding.
References: #647, parent #633, vocabulary #646 / PR #658.

## Ownership and access

Seven tables live in packages/db/src/schemas/job-board.ts, with relations registered
in the existing client. Enums reuse JOB_BOARD. API owns matching, normalization,
transactions and access; Cron orchestrates adapters/delivery; Blade owns interfaces.
No API/Cron/UI behavior is implemented here. Career reads require sign-in and a
Member profile. Members mutate only their applications/preferences. Community DTOs
exclude private applications and never select privateNotes. Officer changes require
explicit control permissions and audits in the later API/settings issue. The DB
schema does not enforce end-user authorization.

## Tables

| Table | Identity and state |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Job | Unique identityKey; company label/key and optional Company FK; title/track/category/URL/description/compensation/arrangement; posted/first-seen/last-seen dates; active/inactive state; single Discord destination, frozen delivery track, receipt and retry/claim state |
| JobSource | Unique provider/boardKey; label, optional Company FK, enabled flag, polling interval/checkpoints, baseline completion, conditional-request validators and claim/error state |
| JobObservation | Unique source/external ID; Job FK; source URL/data, posted/first/last-seen dates and inactivity evidence |
| JobLocation | Job FK; original text; city/state/remote_us/unknown kind; Census city key and state |
| JobAlertPreference | Unique member/track/kind/targetKey; optional original custom value |
| JobApplication | Unique member/job; stage/round/visibility/private notes; timestamps |
| JobDigest | Unique track/scheduled instant; frozen window/destination; receipt and retry/claim state |

Member gets two non-null, false-default digest opt-in booleans. All seven tables
count against the limit; no join-table exception. Reuse DiscordConfig for channel
and role IDs in the later settings implementation, with no routing changes here.

Provider is a code-owned adapter key. boardKey is officer-managed feed/board identity.
New providers require code; existing supported boards can be registered later via
Blade. Sources default disabled; jobs default suppressed. No seed or runtime import.
Polling baselines and source health live on JobSource, not a generic config store.

API computes identityKey from a namespaced employer requisition ID or normalized
employer apply URL; unknown identities use namespaced source/external identity.
Never match on company/title alone. Unique identities protect concurrent inserts;
cross-source matching and conflict retries require API code. Create/find a job and
link its observation in one transaction. Source/observation history is retained;
disable sources and inactivate jobs rather than deleting them.

Keep companyLabel/companyKey even if Company resolution fails. Do not automatically
create/approve Guild companies. Company FKs SET NULL on delete; future reads resolve
mergedIntoCompanyId and merge integration needs API tests. Existing moderation is
unchanged. Locations use existing NN-NNNNN Census keys. City requires city/state;
state requires state alone; remote/unknown have neither. Preserve original text;
remote alone is not U.S. evidence. Catalog membership is API validation. Multiple
source descriptions may resolve to the same city.

Alert targetKey is a normalized company key, Census city key, remote_us or normalized
custom location text. customValue preserves Other input. API normalization dedupes
semantically equivalent targets. Applications reference Member.id; deleting a member
cascades preferences/applications. Jobs with applications cannot be deleted. Rounds
are integers 1..10 only for interviewing. Notes are capped at 2000 characters.

## Delivery contracts for follow-up work

Sent requires message ID and timestamp. Sending requires a claim; token/expiry are
paired. Non-suppressed job delivery requires channel and frozen delivery track.
Workers atomically claim rows and fence completion by token. Discord calls happen
outside DB transactions. Expired sending becomes uncertain: reconcile a deterministic
job/digest marker before retrying. Schema alone does not guarantee exactly-once sends.
A sent job is never requeued merely because its classification or channel changes.

Digests use a row per track/scheduled instant, even if routing changes. Select sent
jobs by frozen delivery track/channel and deliveredAt in (windowStart, windowEnd].
Window end cannot exceed scheduledAt. Freeze boundaries before sending; suppress
empty windows. Preserve daily 4 PM America/New_York behavior in the future worker.
Member flags represent desired digest role subscriptions; role reconciliation uses
User.discordUserId, never Member.discordUser (a username).

Posted dates remain separate from discovery. Closure requires explicit evidence or
absence from a successful complete employer inventory. Rolling-feed disappearance
and failures do not close observations. API aggregates trustworthy observation
activity. No closure, track-selection or delivery triggers are introduced.

## Migration and verification

Generate additive SQL, snapshot and journal. Validate only in disposable local DBs.
No existing job/preference import or production migration. Existing member opt-ins
become false. Application rollback leaves unused storage intact; never drop user
history casually. Root format/lint/typecheck cover affected consumers; run DB tests
and validator regressions. No React changes. User approval authorizes local work;
GitHub #633 review remains separately recorded, not claimed complete.

### Identifier fingerprints

Job.identityHash, JobSource.boardKeyHash and JobObservation.externalIdHash are
stored, database-generated SHA-256 hex fingerprints of the original UTF-8 keys.
Unique constraints target the fingerprint, with provider/sourceId where applicable,
so long URLs and provider identifiers do not exceed PostgreSQL B-tree limits.
The original values remain intact. Future writers must use these columns as
Drizzle conflict targets and verify the original key matches before updating a
conflicting row; a mismatched key is an error, never permission to overwrite.
Indexed lookups should compute the same SHA-256 fingerprint and compare the raw
key as well. No ingestion/upsert service is implemented in this database-only work.
Migration 0064 creates the fingerprint columns and their unique constraints directly.
151 changes: 151 additions & 0 deletions .forge/features/career-job-storage/status.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Career Job Storage Status

Current phase: Squashed migration validated; awaiting dev-lead schema approval and PR review.

## Decisions

- 2026-11-03: User approved database-only #647 after reverse prompting: seven tables,
one post per job, supported-source configuration, Member digest opt-ins and JobDigest.
- Employer track evidence wins; fallback is first accepted track. No repost on
reclassification. Rolling-feed disappearance cannot close a job.
- User requested following through with the agreed plan, authorizing local schema
and migration work. No production database writes.
- Branch: db/career-job-storage. Existing .claude/settings.local.json is unrelated.

## Progress

- [x] Reverse-prompt scope and decisions; document agreed design and test scenarios.
- [x] Complete schema and relations.
- [x] Generate and review additive migration.
- [x] Run isolated database tests and validator regressions.
- [x] Run root checks, inspect diff and prepare handoff.

## Validation

- Feature generator passed. Git refs and pnpm store required sandbox escalation.
- Use pnpm.cmd: PowerShell blocks pnpm.ps1. Node is 25.8.0; repo wants ^24.22.1.
- `pnpm.cmd typecheck`: passed, 33 tasks. Adding Member fields required two
Blade fixture updates and excluding digest fields from the profile form source
type; no rendered UI or form behavior changed.
- `pnpm.cmd lint`: passed, 31 tasks, warnings only (including existing large
files and the new integration-test describe block).
- `pnpm.cmd --filter @forge/db with-env vitest run`: 166 passed, 2 failed.
One failure was the expected migration-count update, now fixed and verified.
The other is the unchanged historical SQL checksum described below.
- Final `pnpm.cmd --filter @forge/db with-env vitest run
src/tests/job-board-migration.test.ts src/tests/migration-lineage.test.ts`:
10 passed, 1 failed. All 9 new storage tests and the updated journal/snapshot
lineage test pass. Fresh migration and populated-table upgrade run on disposable
databases, including concurrent inserts through independent pool connections.
- Remaining checksum failure: `0010_wooden_supreme_intelligence.sql` is CRLF in
this Windows checkout (SHA-256 starts e5285213); Git HEAD and LF-normalized bytes
both match the expected a620ea7e hash. Historical SQL files were not changed.
- `pnpm.cmd --filter @forge/validators exec vitest run
src/tests/job-board.test.ts`: 30 passed.
- Three targeted Blade files (member-dashboard, member-profile-settings-form,
member-profile-form-values): 19 tests passed.
- `pnpm.cmd format`: failed across 24 packages on existing checkout formatting.
Untouched `packages/consts/src/career.ts` fails Prettier with its CRLF bytes and
passes when checked in memory with LF only. No repo-wide formatting changes.
- Explicit Prettier check of all changed source, test, document and metadata
files: passed. `git diff --check`: passed.
- `pnpm.cmd --filter @forge/db with-env drizzle-kit generate`: final check
reports no schema changes. Generated `0064_career_job_storage.sql`, snapshot and
journal are aligned. Original line endings restored on unchanged older snapshots
after the generator's broad metadata formatter; no historical content changes.

## Handoff

- Renamed the uncommitted migration to `0064_career_job_storage.sql` at the
user's request; updated journal and test references. Targeted rerun: 10 passed,
with the unrelated historical checksum test excluded (1 skipped).
- Seven tables, eight enums, two false-default Member columns, query relations,
named uniqueness/check constraints and search/delivery indexes are implemented.
- Design and follow-up runtime/access contracts are in srd.md. No current app
sends jobs, syncs digest roles, or accepts source settings through this change.
- Migration is committed and pushed in PR #666; no production/shared database was
migrated.
- Root formatting and historical checksum remain checkout limitations. GitHub
review in #633 is still outstanding.

## Linux contribution review

- Ran the Forge standard review with migration, test-quality and placement
reviewers after a green static gate. Used an isolated LF copy in Docker with
Node 24.22.1 and PostgreSQL 16, matching CI; only .env.example placeholders and
disposable database credentials were used. The original checkout was preserved.
- `pnpm verify:precommit`: passed (React changed-file analysis, root formatting,
31 lint tasks and 33 typecheck tasks). Cold API lint/build exceeded Node's default
heap; retry used NODE_OPTIONS=--max-old-space-size=6144 and TURBO_CONCURRENCY=1.
- `pnpm build`: passed, 21 tasks. `pnpm lint:ws` also passed during installation.
- Full non-DB CI command, `pnpm exec turbo run test --filter='!@forge/db'`:
passed, 28 tasks, including 1,134 API tests and 911 Blade tests. Other consumers
also passed (2026: 190, Club: 11, Cron: 62, Guild: 18). Final modified DB test
file passed its targeted lint and formatting checks after the review fixes.
- `pnpm db:migrate` applied the complete migration chain successfully; running it
a second time succeeded without pending changes.
- Full DB suite: 168 tests across 31 files passed on LF/PostgreSQL 16. The first
concurrent run hit an unrelated 5-second event-workflow timeout; a one-worker
retry passed without changing that test or increasing its timeout.
- Fixed a real CI portability issue: PostgreSQL 16 reports 23503 for the protected
job deletion while PostgreSQL 18 reports 23001. The assertion now checks the
specific application foreign-key constraint instead of the version-dependent code.
- Fixed a masked receipt test: set a valid destination before asserting that sent
jobs require receipts, assert the receipt constraint by name, and test both
one-sided receipt cases. All 9 storage tests passed again after this change.
- No actionable placement/current-consumer findings. Existing Guild projections
do not expose application notes or the new digest fields; profile updates do
not overwrite digest preferences.

### Resolved P2: unbounded keys in unique B-tree indexes

Review reproduced index-size failures for long Job.identityKey, JobSource.boardKey
and JobObservation.externalId values on PostgreSQL 16. User requested continuing
with the fix. The squashed migration 0064 creates stored SHA-256 fingerprint
columns and the three unique constraints directly, preserving original keys.
Named generated columns support typed Drizzle conflict targets. The immutable
conversion preserves Unicode and literal backslashes. Future writers must verify
raw-key equality before updating a fingerprint conflict; ingestion is out of scope.

Regression coverage includes incompressible 8KB identifiers, Unicode/backslashes,
duplicate upserts, distinct identifiers, and preservation of existing member/company data.

Final validation after the fix:

- `pnpm verify:precommit` passed in the isolated LF/Node 24.22.1 environment:
React analysis, format (24 tasks), lint (31 tasks), typecheck (33 tasks).
- Full PostgreSQL 16 DB suite: 169 tests across 31 files passed.
- `pnpm db:migrate` passed twice against the isolated DB; Drizzle generation found
no schema drift.
- Normalized checksums confirm all changed app/package files match the tested copy.
- Standard review: migration and test reviewers rechecked the fix with no remaining
blockers; the earlier placement review remains applicable.
- Full build and non-DB suites passed earlier in this review; after the fingerprint
change, the root static gate and complete DB suite were rerun.
- No new tables, dependencies, or CI-setting changes in the identifier fix.
GitHub #633 design approval remains outstanding. No push or production migration.

## Links

- https://github.com/KnightHacks/forge/issues/647
- https://github.com/KnightHacks/forge/issues/633
- https://github.com/KnightHacks/forge/pull/658
- No GitHub comment, PR or lead review has been recorded by this session.

## Migration squash follow-up

User explicitly requested squashing 0064/0065 and clarifying the hash helper.
Only disposable local databases were migrated by this session; no shared deployment
or push occurred. Regenerated one descriptive 0064 migration and snapshot from
0063, updated the journal/lineage assertions and single-migration upgrade test,
and removed the obsolete intermediate-state backfill test. Hash behavior and
seven-table scope are unchanged. The helper explains the B-tree entry limit,
literal backslash handling, and generated-column immutability requirement.
Validation passed: 12 focused migration/lineage tests on PostgreSQL 16; fresh
migration and idempotent rerun; targeted formatting/lint, DB typecheck, and no
Drizzle schema drift. The user additionally confirmed that root lint, root
typecheck, DB tests, and db:generate ran after the squash. The local 0010 lineage
check has a CRLF-only failure; the isolated LF lineage suite passed.
Committed as one commit rebased onto main and opened as PR #666. Dev-lead schema
approval is still outstanding. Optional follow-ups: delivery_track digest index coverage and
an automated schema-drift regression test.
15 changes: 15 additions & 0 deletions .forge/features/career-job-storage/test-cases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Career Job Storage Test Cases

- TC01: Fresh migration creates seven tables; sources default disabled and jobs suppressed.
- TC02: Concurrent duplicate job and observation identities yield one successful insert; multiple sources may reference one job.
- TC03: City/state/remote/unknown locations retain text. Reject malformed keys and contradictory shapes.
- TC04: One application per member/job; enforce stage/round bounds. Inactivity preserves history and deletion cannot remove referenced jobs.
- TC05: Sent job/digest records require receipts; reject malformed IDs, unpaired leases and sending without a claim.
- TC06: Duplicate digest schedule fails despite destination changes; reject reversed/future windows.
- TC07: Alert identity is unique per member/track/kind/target; member deletion cascades personal records.
- TC08: Upgrade pre-feature member/company fixtures; preserve values and default both opt-ins false.
- TC09: Query new relations through the DB client; all consumers typecheck and validators regress cleanly.
- TC10: Preserve incompressible long identifiers, Unicode and literal backslashes; generated fingerprints match UTF-8 SHA-256 and typed upserts resolve duplicates. Distinct job identifiers remain distinct.

API privacy, matching, closure, authorization, role sync and actual Discord recovery
are follow-up tests, not claims made by this schema implementation.
2 changes: 2 additions & 0 deletions apps/blade/src/lib/member-profile-form-values.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ export type MemberProfileFormSource = Omit<
| "currentCityKey"
| "gender"
| "guildLocationVisible"
| "jobInternshipDigestEnabled"
| "jobNewGradDigestEnabled"
| "levelOfStudy"
| "major"
| "raceOrEthnicity"
Expand Down
Loading
Loading