Name webhook trigger's URL at configuration time by elias-ba · Pull Request #5105 · OpenFn/lightning

elias-ba · 2026-08-30T03:06:30Z

Description

This PR lets you name a webhook trigger's URL path, so you know the endpoint before you deploy. A trigger with a path of facility-001 in project abc-123 answers at /i/abc-123/facility-001. You can set it in the trigger panel, in project.yaml, or through the workflows API.

This came from deploying the same workflows to roughly a hundred facilities. Today that means deploying first, then opening a hundred workflows and copying a hundred generated UUIDs by hand. Now the names go in the deployment templates, and the integrators get the full list before anything is deployed.

The path namespaces on the project id rather than the project name, which is what we agreed in the requirements session. Project names are mutable and not unique, so a name in a URL is a contract that breaks on a rename. Namespacing on the project also means a path only has to be unique within its project, so a sandbox clone keeps the parent's path and answers on its own project's URL, which is what #4937 asks for.

Every URL already in the wild keeps working. /i/<trigger-id> is tried first and never changes. Paths that were addressable bare at /i/<path> before this are marked by the migration and keep answering there, and that set can only shrink.

The workflow version hash now includes the path, so a workflow whose only change is its endpoint name is no longer reported as unchanged. A workflow without a path hashes exactly as it did, and @openfn/project hashes the same key in OpenFn/kit#1511, so the two stay in step.

Closes #4952.

Validation steps

Open a workflow, click the webhook trigger, click Edit. Type facility-001 into Custom URL path and click Finish. The webhook URL on the show panel becomes /i/<project-id>/facility-001.
curl -X POST localhost:4000/i/<project-id>/facility-001 -d '{}' -H 'content-type: application/json' creates a work order. So does the old /i/<trigger-id> URL.
Type Orders Intake in the field. It tells you it will be named orders-intake. Type !!! and it goes red and refuses to save. Type a name another workflow in the project already uses and the save reports it against the field.
Export the project to YAML. The trigger carries custom_path. Deploy it back with openfn deploy (needs CLI: support custom webhook paths in project.yaml kit#1511) and the path survives.

Additional notes for the reviewer

triggers.custom_path has existed since many years now with no validation, no uniqueness and no UI, and the lookup resolved it with coalesce(custom_path, id) through Repo.one. Any user could point a path in their own project at another project's trigger id and permanently 500 that project's live endpoint. I reproduced it before fixing it. Production has no custom_path rows; self-hosted instances may.
The migration's data steps have no test, because there is no migration-test harness in the repo.

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just want to know!):

I have used Claude Code
I have used another model
I have not used AI

You can read more details in our Responsible AI Policy

Pre-submission checklist

I have performed an AI review of my code (we recommend using /review with Claude Code)
I have implemented and tested all related authorization policies. (e.g., :owner, :admin, :editor, :viewer)
I have updated the changelog.
I have ticked a box in "AI usage" in this PR

github-actions · 2026-08-30T03:11:06Z

Based on my review of the diff (custom webhook path feature — namespaced by project, with grandfathered bare paths), all three checks pass or are N/A.

Security Review ✅

S0 (project scoping): Workflows.get_webhook_trigger/2 filters by_project_path/3 on both project_id and custom_path (lib/lightning/workflows.ex:996-998); a composite FK on (workflow_id, project_id) (migration line 34) prevents the denormalized project_id from drifting, Trigger.new/1 and cast_changeset/2 strip server-owned fields (lib/lightning/workflows/trigger.ex:88), and the provisioner rejects project_id/legacy_bare_path from API params (lib/lightning/projects/provisioner.ex:770-782).
S1 (authorization): N/A — no new controller actions, LiveView events, or channel handlers; webhook ingest remains gated by per-trigger webhook_auth_methods (existing pattern), and trigger edits still flow through the existing provisioner and collaborative-editor authorization paths.
S2 (audit trail): custom_path is included in @trigger_fields for snapshots (lib/lightning/workflows/snapshot.ex:117-124) so trigger changes are captured via the existing workflow-snapshot mechanism; mark_for_deletion still emits Audit.marked_for_deletion in the same transaction that clears paths (lib/lightning/workflows.ex:849-855), matching how other trigger config fields are audited.

codecov · 2026-08-30T03:14:50Z

Codecov Report

❌ Patch coverage is 98.24561% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.7%. Comparing base (0097126) to head (66b8c41).

Files with missing lines	Patch %	Lines
lib/lightning/workflows/trigger.ex	95.7%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #5105     +/-   ##
=======================================
+ Coverage   90.6%   90.7%   +0.1%     
=======================================
  Files        422     422             
  Lines      20039   20147    +108     
=======================================
+ Hits       18164   18274    +110     
+ Misses      1875    1873      -2

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

A webhook trigger can now hold a `custom_path`, so its endpoint can be named rather than only addressed by the trigger's generated id. The name has to be unique within a project rather than across the instance, so triggers carry a `project_id` denormalised from their workflow. A composite foreign key on `(workflow_id, project_id)` stops it drifting from the workflow's real project, and it is resolved inside the insert transaction so no caller has to remember it. It is nullable because it only scopes a path. A path is validated as a URL segment: lowercase letters, digits, hyphens and underscores, at most 255 characters, blank meaning unset, and never UUID-shaped, since those resolve against trigger ids and would be unreachable. The column has existed since 2022 with no validation, no uniqueness and no UI, so instances can already hold values these rules reject. The migration backfills the project, releases paths held by workflows deleted since, marks the triggers that already answered at a bare URL, dedupes what was never unique, and only then adds the indexes. Its data steps are gated on the trigger being disabled: webhook ingest ignores `deleted_at`, so a hidden workflow with an enabled trigger is still serving.

`/i/<project-id>/<custom-path>` now reaches a webhook trigger, alongside the generated `/i/<trigger-id>` URL that every trigger has always had. Resolution is tried in order and no step can match more than one row, so a request can never fail on an ambiguous path. Each step is backed by a constraint: the primary key for a trigger id, and a partial unique index for each of the two path forms. Before this the lookup resolved a path with `coalesce(custom_path, id)` through `Repo.one`, so a path set in one project could match another project's trigger id, return two rows and permanently 500 that project's live endpoint. Paths that were addressable bare at `/i/<path>` keep answering there. That set is fixed by the migration and never grows, so a path created from now on cannot claim a bare URL out from under an existing one. The ingest plug's telemetry context is narrowed to the first path segment, which is always the address. The rest is caller data: `/i/<trigger-id>/Patient/123` carries record identifiers.

A path set in the app now survives a pull, and one written in `project.yaml` now deploys, so a project can be described end to end without opening the editor. An absent key means unchanged, so deploying a spec written before this existed cannot wipe a path. `project_id` and the bare-URL marker are refused in a trigger payload: they are server-owned, and `validate_extraneous_params/2` builds its allow-list from the schema, so adding them as fields would have silently started accepting them. Provisioning releases the paths of workflows that are hidden and disabled, both before the write, so one document can drop a workflow and claim its path in the same breath, and after it project-wide, since a checked-in yaml can still name the path of a workflow deleted since. Export, the JSON endpoint and the merge all drop a path the current rules would reject, so a document written before them cannot fail the import it is deployed into. A sandbox clone keeps its parent's path verbatim and answers on its own project's URL, so promoting one never moves the parent's live URL.

The webhook wizard gains a Custom URL path field. Typing a name shows what it will be stored as, the way the project form treats a project name, and the show panel's URL updates once the trigger is saved. The field only judges a path the user actually changed, so a value written before these rules existed does not block edits to anything else. Becoming a webhook counts as a change, since a path that meant nothing on a cron row is about to become a live URL and the server will check it. A path the server refused is not the trigger's, and for a duplicate it resolves to whichever workflow legitimately owns the name, so the panel falls back to the generated URL rather than offering one that posts into another workflow. The same goes for a legacy path holding a slash, which is stored verbatim but is not addressable. Applying a YAML or AI answer that never mentions the path keeps what the trigger holds, matching the way the provisioner reads an absent key.

OpenFn/lightning#5105 lets a webhook trigger be named, so its URL is known before it is deployed. Deploy never sent the field. The new-trigger path builds its payload from an allowlist that does not include it, and the existing-trigger path builds its fields explicitly. A path only travels when the spec mentions it. An absent key means the server keeps what it has, so deploying a spec written before this existed does not wipe a path set through the app. Writing it blank clears it, either as an empty string or as a bare key, which YAML reads as null. Cron and kafka triggers ignore it, matching the server. Both insertion points decide on the resolved trigger type rather than on what the spec wrote, because type is optional in a spec and defaults to webhook. A trigger that names a path and nothing else is a legal spec, and guarding on the spec's own type would drop the path without saying so. Pull needs no change. It fetches the yaml from Lightning, which now exports the field.

A workflow whose only change was its endpoint name hashed the same as before, so anything comparing versions decided nothing had changed. The CLI's find-changed-workflows skipped it and openfn project deploy sent nothing. Costs existing workflows nothing. A nil serialises to an empty string and the parts are concatenated, so hashing the key does not move the hash of a workflow that has no path. @openfn/project skips undefined values for the same reason, which is what keeps the two implementations equal, and its parity test against a real Lightning hash still passes. Paired with OpenFn/kit#1511, which adds the key on that side.

…make Two gaps the review left open. The workflow version hash ignored custom_path, so find-changed-workflows saw no change when a webhook was renamed and openfn project deploy skipped it. Lightning hashes the same key now, in OpenFn/lightning#5105, and the parity test against a real Lightning hash still passes: both sides leave a workflow without a path hashing exactly as it did. And deploy diffs the server's project against the payload it is about to send, so every trigger field held on the server and never named in the spec showed up as a removal that was not going to happen. Absence means leave it alone, so the diff now hides what the payload does not carry. webhook_reply and webhook_response_config had this already.

Two problems with hashing custom_path verbatim, both found in review. ProvisioningJSON and ExportUtils drop a path the naming rules reject, so the CLI never receives one. Hashing it here left the app hash and the CLI hash disagreeing forever for exactly the grandfathered rows the migration exists to keep working, and every openfn project diff would have reported the workflow as changed with nothing to show for it. And Map.take is type blind, so a stale path on a cron or kafka row, which never served a URL and was never validated, moved the hash of a workflow nobody had touched. Both are gated now, and both directions are covered by a test that fails when the gate is removed.

custom_path already has a stricter format rule and a length cap from #5105, so the null-byte and width guards this branch added to it are dead weight and the assertions were checking for a message that no longer surfaces. The invisible-codepoint fixture gains U+0890, U+0891, U+08E2 and U+110CD. Erlang 28 brought PCRE2 with newer Unicode tables, which classify those four as Format where the old build did not. For the same reason the client is no longer stricter than the server on that set, so the test that pinned the four-codepoint skew now pins zero.

* Carry a webhook's custom path through deploy OpenFn/lightning#5105 lets a webhook trigger be named, so its URL is known before it is deployed. Deploy never sent the field. The new-trigger path builds its payload from an allowlist that does not include it, and the existing-trigger path builds its fields explicitly. A path only travels when the spec mentions it. An absent key means the server keeps what it has, so deploying a spec written before this existed does not wipe a path set through the app. Writing it blank clears it, either as an empty string or as a bare key, which YAML reads as null. Cron and kafka triggers ignore it, matching the server. Both insertion points decide on the resolved trigger type rather than on what the spec wrote, because type is optional in a spec and defaults to webhook. A trigger that names a path and nothing else is a legal spec, and guarding on the spec's own type would drop the path without saying so. Pull needs no change. It fetches the yaml from Lightning, which now exports the field. * Keep a webhook's custom path top-level when reading app state `from-app-state` destructures the trigger keys it knows and sweeps the rest into `openfn:`, so a pulled `custom_path` landed nested there rather than beside `webhook_reply`. The v1 and v2 project formats would have disagreed about where a webhook's path lives. The workflow version hash is deliberately left alone. Lightning's `canonical_form/1` does not hash `custom_path` either, and adding it on one side only would make the two hashes disagree for every workflow holding a path, which is what `find-changed-workflows` compares. Moving it needs both sides at once. * Hash the custom path, and stop deploy reporting removals it will not make Two gaps the review left open. The workflow version hash ignored custom_path, so find-changed-workflows saw no change when a webhook was renamed and openfn project deploy skipped it. Lightning hashes the same key now, in OpenFn/lightning#5105, and the parity test against a real Lightning hash still passes: both sides leave a workflow without a path hashing exactly as it did. And deploy diffs the server's project against the payload it is about to send, so every trigger field held on the server and never named in the spec showed up as a removal that was not going to happen. Absence means leave it alone, so the diff now hides what the payload does not carry. webhook_reply and webhook_response_config had this already. * Show what a deploy will clear, and hash a path only where it means something Three things from review. The diff mask hid fields the server was about to wipe. validate_by_type clears a path when a webhook becomes a cron, and clears the response config when the reply mode stops being after_completion, whether or not the payload names them. Masking those made the user confirm a deploy that retires a live URL with nothing in the diff saying so. The mask now stands down when the type or the reply mode is changing, and for a trigger being deleted, where the diff is the only place the user sees what goes with it. A path on a cron or kafka trigger is no longer hashed. It never served a URL, and Lightning stopped hashing one there in the paired PR, so the two would otherwise disagree. And the type check: the fixture's inferred literal type has no custom_path, so the new test needed the cast the rest of that file already uses. ava does not type check, and I ran it instead of tsc. * Narrow the spec trigger instead of asserting it Review point. The non-null assertion says the value is there without anything enforcing it. Nothing can reach this line with an undefined spec trigger today, since splitZip walks the union of keys and the two branches above return for spec-only and state-only, but the check costs nothing and does not depend on that staying true. * One changeset per package Deploy had two, a minor for the feature and a patch for the diff fix. The tool flattens them to the higher bump anyway, and one entry per package per release is the shape everywhere else. * Bump versions deploy 0.15.0, project 0.19.0, lexicon 2.4.3, and the dependents changesets pulls along with them. * Cut the changelog entries to a sentence CLAUDE.md says a changeset note is a single short high-level sentence with no implementation detail. Mine were three paragraphs of it. * update canonical test --------- Co-authored-by: Joe Clark <jclark@openfn.org>

custom_path already has a stricter format rule and a length cap from #5105, so the null-byte and width guards this branch added to it are dead weight and the assertions were checking for a message that no longer surfaces. The invisible-codepoint fixture gains U+0890, U+0891, U+08E2 and U+110CD. Erlang 28 brought PCRE2 with newer Unicode tables, which classify those four as Format where the old build did not. For the same reason the client is no longer stricter than the server on that set, so the test that pinned the four-codepoint skew now pins zero.

github-project-automation Bot added this to Core Aug 30, 2026

github-project-automation Bot moved this to New Issues in Core Aug 30, 2026

elias-ba mentioned this pull request Aug 30, 2026

CLI: support custom webhook paths in project.yaml OpenFn/kit#1511

Merged

3 tasks

elias-ba added the v2.19.0 label Aug 30, 2026

elias-ba changed the title ~~Let users name a webhook trigger's URL at configuration time~~ Define webhook trigger's URL at configuration time Aug 30, 2026

elias-ba changed the title ~~Define webhook trigger's URL at configuration time~~ Name webhook trigger's URL at configuration time Aug 30, 2026

elias-ba requested review from AishaHassen, brandonjackson and doc-han August 30, 2026 13:54

elias-ba added 5 commits August 30, 2026 14:36

Add the changelog entry

12dfdb7

elias-ba force-pushed the fix/4952-webhook-custom-path branch from 9359f11 to 12dfdb7 Compare August 30, 2026 14:37

This was referenced Aug 30, 2026

Support custom webhook paths in project.yaml OpenFn/kit#1516

Closed

Stable webhook endpoint naming on promote #4937

Open

doc-han approved these changes Aug 31, 2026

View reviewed changes

elias-ba merged commit ac60463 into main Aug 31, 2026
7 checks passed

elias-ba deleted the fix/4952-webhook-custom-path branch August 31, 2026 19:18

github-project-automation Bot moved this from New Issues to Done in Core Aug 31, 2026

elias-ba mentioned this pull request Sep 2, 2026

List, edit, and delete a webhook trigger's custom URLs #5111

Merged

7 tasks

elias-ba mentioned this pull request Sep 4, 2026

Workflow chat does not know about webhook custom paths OpenFn/apollo#669

Open

elias-ba mentioned this pull request Sep 4, 2026

Document webhook custom URLs OpenFn/docs#849

Open

3 tasks

Navigation Menu

Uh oh!

Name webhook trigger's URL at configuration time - #5105

Name webhook trigger's URL at configuration time#5105
elias-ba merged 7 commits into
mainfrom
fix/4952-webhook-custom-path

elias-ba commented Aug 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 30, 2026

Uh oh!

codecov Bot commented Aug 30, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

elias-ba commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Validation steps

Additional notes for the reviewer

AI Usage

Pre-submission checklist

Uh oh!

github-actions Bot commented Aug 30, 2026

Security Review ✅

Uh oh!

codecov Bot commented Aug 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

elias-ba commented Aug 30, 2026 •

edited

Loading

codecov Bot commented Aug 30, 2026 •

edited

Loading