diff --git a/.arcane-auditor/README.md b/.arcane-auditor/README.md new file mode 100644 index 0000000..5fa468b --- /dev/null +++ b/.arcane-auditor/README.md @@ -0,0 +1,94 @@ +# Arcane Auditor configuration + +This folder configures the example audit that runs on every pull request +(`.github/workflows/audit-examples.yml`) and locally through +`node scripts/audit-examples.mjs`. + +| File | Purpose | +| --- | --- | +| `config.json` | Rule configuration for [Arcane Auditor](https://github.com/Developers-and-Dragons/ArcaneAuditor). Generated with `ArcaneAuditorCLI generate-config`, then adjusted (see below). | +| `action-ref` | The Arcane GitHub Action revision the CI workflow uses, as `owner/repo@ref`. `scripts/install-arcane.sh` reads it so local installs match CI. | +| `bin/` | Local CLI install, gitignored. Created by `scripts/install-arcane.sh`. | + +The hub's own rules (folder naming, `example.json`, README sections, period +literals, app reference ids) live in `scripts/audit/hub-rules.mjs`, not here. + +## Rule policy + +Arcane ships 48 rules in two tiers. ACTION findings fail the audit check, +ADVICE findings are shown as suggestions. `config.json` keeps every rule +enabled and changes three things for a public examples hub: + +| Rule | Change | Why | +| --- | --- | --- | +| `HardcodedApplicationIdRule` | ADVICE to ACTION | Examples exist to be copied. A hardcoded app id guarantees the copy breaks, and the fix is mechanical (`site.applicationId`). | +| `OrchestrationGlobalErrorHandlerRule` | ACTION to ADVICE | Error-handler scaffolding is not always the lesson an orchestration example teaches. Worth suggesting, not worth blocking. | +| `OrchestrationApiStepErrorHandlerRule` | ACTION to ADVICE | Same reasoning. | +| `PMDSectionOrderingRule` | fix strategy to `human_review` | Arcane v2.0.0 has no automatic fix payload for this rule; marking it actionable produced empty suggestions. | + +Everything else runs at Arcane's default severity. Prefer downgrading a rule +over disabling it, so the best-practices doc can still explain it. + +Policy applied on top by `scripts/audit/report.mjs`: + +- `catalog/` folders are held to the stricter bar: ADVICE counts as ACTION. +- In a folder that already existed, ACTION findings on lines the PR did not + touch are downgraded so contributors are only blocked on what they wrote. +- The `audit-override` label, or the repository variable `AUDIT_MODE=advisory`, + turns the whole check advisory. + +## Bumping Arcane + +1. Pick the release on the Arcane releases page and note its Linux and macOS + CLI asset hashes (`sha256sum` the downloads yourself). +2. In the Arcane action repository, add the hashes to + `.github/action/install.sh` and bump the default `version` in `action.yml`. + Push, note the commit sha. +3. Update `action-ref` here and the `uses:` line in + `.github/workflows/audit-examples.yml` to that sha. +4. Regenerate `config.json` if the rule set changed: + `ArcaneAuditorCLI generate-config -o /tmp/new.json`, diff against the + current file, and re-apply the overrides above. +5. Re-run the regression check below. + +## Regression check + +Pull request [#7](https://github.com/Workday/WorkdayDeveloperProgram/pull/7) +(`examples/Promotion_Nomination`) is the reference case: it contains most of +the mistakes the audit exists to catch. To re-run it: + +```bash +git fetch origin pull/7/head:pr-7 +git worktree add /tmp/pr-7 pr-7 +cp -R scripts .arcane-auditor/config.json /tmp/pr-7/ # bring the current audit scripts along +cd /tmp/pr-7 && node scripts/audit-examples.mjs --changed main HEAD +``` + +Expected with Arcane v2.0.0 and the current hub rules (62 findings, 32 +blocking): + +Hub rules + +- `HubFolderKebabCaseRule`: `Promotion_Nomination` +- `HubExampleJsonRule`: `example.json` missing (hint: rename `app-info.json`), and `app-info.json` line 1 is the literal word `JSON` +- `HubReadmeSectionsRule`: README is raw HTML, none of the four sections +- `HubGitkeepRule`: `model/.gitkeep`, `presentation/.gitkeep` +- `HubHardcodedPeriodLiteralRule`: `"2026-Q1"` in `managerNomination.pmd` +- `HubAppReferenceIdRule`: `promotionNomination_rvylxm` in the `.amd` and `.smd` + +Arcane, ACTION + +- `HardcodedWorkdayAPIRule`: 3 endpoints across the two PMDs, 5 data providers in the `.amd` +- `HardcodedApplicationIdRule`: the `.amd` data provider and the `submitPromotion` URL +- `EndpointFailOnStatusCodesRule`: 8 endpoints +- `WidgetIdRequiredRule`: 7 widgets +- `ScriptConsoleLogRule`: 2 live `console.info` calls (the commented-out ones do not fire) + +Arcane, ADVICE + +- `ScriptVarUsageRule` (6), `ScriptStringConcatRule` (8), `EndpointBaseUrlTypeRule` (4), `StringBooleanRule` (2), `PMDSectionOrderingRule` (2), `ScriptComplexityRule` (1), `EndpointNameLowerCamelCaseRule` (1) +- `ArcaneAuditorWarning`: Arcane's script parser gives up on one block in `managerNomination.pmd` (`var responseEmpData =getEmployeeData.invoke(`), so script rules are skipped for that block. That is an upstream grammar gap worth reporting. + +Of the actionable findings, the hardcoded URL, console, `var`, and string +boolean ones render as one-click suggestions on the PR. `failOnStatusCodes` +insertions and multi-line string concatenations render as code blocks. diff --git a/.arcane-auditor/action-ref b/.arcane-auditor/action-ref new file mode 100644 index 0000000..b981561 --- /dev/null +++ b/.arcane-auditor/action-ref @@ -0,0 +1 @@ +Ekwuno/ArcaneAuditor@c31316d1147cb0e2d3f47688e668f7f3b2f9e887 diff --git a/.arcane-auditor/config.json b/.arcane-auditor/config.json new file mode 100644 index 0000000..02502ac --- /dev/null +++ b/.arcane-auditor/config.json @@ -0,0 +1,326 @@ +{ + "rules": { + "ScriptComplexityRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptFunctionParameterCountRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptLongFunctionRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptNestingLevelRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptLongBlockRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptConsoleLogRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptVarUsageRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptDeadCodeRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptVariableNamingRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptFunctionParameterNamingRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptArrayMethodUsageRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptMagicNumberRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptFunctionReturnConsistencyRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptStringConcatRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptVerboseBooleanCheckRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptDescriptiveParameterRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptOnSendSelfDataRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptEmptyFunctionRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptUnusedFunctionRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptUnusedFunctionParametersRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptUnusedVariableRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "ScriptUnusedIncludesRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "EndpointFailOnStatusCodesRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "EndpointNameLowerCamelCaseRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "EndpointOnSendSelfDataRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "EndpointBaseUrlTypeRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "NoIsCollectionOnEndpointsRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "OnlyMaximumEffortRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "NoPMDSessionVariablesRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "WidgetIdRequiredRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "WidgetIdLowerCamelCaseRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "GridPagingWithSortableFilterableRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "HardcodedWorkdayAPIRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "FooterPodRequiredRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "StringBooleanRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "EmbeddedImagesRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "PMDSectionOrderingRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": "human_review", + "custom_settings": {} + }, + "HardcodedApplicationIdRule": { + "enabled": true, + "severity_override": "ACTION", + "fix_strategy_override": null, + "custom_settings": {} + }, + "HardcodedWidRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "PMDSecurityDomainRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "OrchestrationSecurityDomainRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "OrchestrationGlobalErrorHandlerRule": { + "enabled": true, + "severity_override": "ADVICE", + "fix_strategy_override": null, + "custom_settings": {} + }, + "OrchestrationApiStepErrorHandlerRule": { + "enabled": true, + "severity_override": "ADVICE", + "fix_strategy_override": null, + "custom_settings": {} + }, + "OrchestrationBranchOnConditionsNestingRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "OrchestrationVerboseBooleanCheckRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "OrchestratePreferExplicitDefaultAccessor": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "FileNameLowerCamelCaseRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + }, + "MultipleStringInterpolatorsRule": { + "enabled": true, + "severity_override": null, + "fix_strategy_override": null, + "custom_settings": {} + } + }, + "file_processing": { + "max_file_size": 52428800, + "max_zip_size": 524288000, + "relevant_extensions": [ + ".pod", + ".pmd", + ".script", + ".amd", + ".smd", + ".wqlquery", + ".orchestration", + ".suborchestration" + ], + "encoding": "utf-8", + "log_level": "ADVICE", + "chunk_size": 16384, + "max_concurrent_files": 20, + "fallback_encodings": [ + "utf-8", + "latin-1", + "cp1252", + "iso-8859-1" + ] + }, + "output": { + "format": "text", + "include_rule_details": true, + "group_by_file": false, + "sort_by_severity": true, + "max_findings_per_rule": null + }, + "fail_on_severe": false, + "fail_on_warning": false, + "quiet": false +} diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 84be353..54a0bee 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -6,8 +6,9 @@ - [ ] The example lives entirely in its own folder under `examples/` - [ ] `node scripts/validate-examples.mjs --check` passes (valid `example.json`, README present, index table in sync) -- [ ] The README says what the artifact is and how to use it +- [ ] The README says what the artifact is, how to use it, and what to change before deploying it (the "Before you deploy" section) - [ ] No credentials, tenant names, or real personal data anywhere in the folder +- [ ] `node scripts/audit-examples.mjs --changed` passes, or the findings it reports are explained below (see [docs/EXAMPLE_BEST_PRACTICES.md](https://github.com/Workday/WorkdayDeveloperProgram/blob/main/docs/EXAMPLE_BEST_PRACTICES.md)) ## Anything reviewers should know? diff --git a/.github/workflows/audit-comment.yml b/.github/workflows/audit-comment.yml new file mode 100644 index 0000000..1456a79 --- /dev/null +++ b/.github/workflows/audit-comment.yml @@ -0,0 +1,67 @@ +# Posts the example audit results on the pull request: one sticky summary +# comment plus inline review comments with one-click suggestions where the +# fix is mechanical. +# +# This runs separately from audit-examples.yml because that workflow has no +# write token on pull requests from forks. This one is triggered by +# workflow_run, always executes the version of these files on the default +# branch, and only ever reads the audit artifact as data. It never checks out +# or runs anything from the pull request itself. + +name: Audit comment + +on: + workflow_run: + workflows: ["Audit examples"] + types: [completed] + +permissions: + contents: read + actions: read + pull-requests: write + +concurrency: + group: audit-comment-${{ github.event.workflow_run.head_sha }} + cancel-in-progress: true + +jobs: + comment: + if: github.event.workflow_run.event == 'pull_request' && github.event.workflow_run.conclusion != 'cancelled' + runs-on: ubuntu-latest + steps: + # Default branch only: trusted scripts, never the PR head. + - uses: actions/checkout@v7 + + - uses: actions/setup-node@v7 + with: + node-version: 20 + + - name: Download the audit report + id: download + continue-on-error: true + uses: actions/download-artifact@v4 + with: + name: audit-report + path: audit + run-id: ${{ github.event.workflow_run.id }} + github-token: ${{ github.token }} + + - name: Post review and summary comment + if: steps.download.outcome == 'success' + uses: actions/github-script@v7 + env: + HEAD_SHA: ${{ github.event.workflow_run.head_sha }} + HEAD_BRANCH: ${{ github.event.workflow_run.head_branch }} + HEAD_OWNER: ${{ github.event.workflow_run.head_repository.owner.login }} + AUDIT_CONCLUSION: ${{ github.event.workflow_run.conclusion }} + with: + script: | + const { postReview } = await import(`${process.env.GITHUB_WORKSPACE}/scripts/audit/post-review.mjs`); + await postReview({ + github, context, core, + reportPath: "audit/report.json", + headSha: process.env.HEAD_SHA, + headBranch: process.env.HEAD_BRANCH, + headOwner: process.env.HEAD_OWNER, + checkConclusion: process.env.AUDIT_CONCLUSION + }); diff --git a/.github/workflows/audit-examples.yml b/.github/workflows/audit-examples.yml new file mode 100644 index 0000000..0d20a39 --- /dev/null +++ b/.github/workflows/audit-examples.yml @@ -0,0 +1,99 @@ +# Audits the example folders a pull request touches with Arcane Auditor plus +# the hub's own rules (scripts/audit-examples.mjs). Findings show up as +# annotations and in the job summary, which works for pull requests from +# forks because nothing here needs a write token. The follow-up workflow +# (audit-comment.yml) posts the same findings as a PR comment with one-click +# suggestions. +# +# ACTION findings fail this check. ADVICE findings never do. To ship a PR +# with open ACTION items, a maintainer adds the "audit-override" label. +# Set the repository variable AUDIT_MODE=advisory to make the whole check +# non-blocking. + +name: Audit examples + +on: + pull_request: + paths: + - "examples/**" + - "catalog/**" + - "scripts/**" + - ".arcane-auditor/**" + - ".github/workflows/audit-examples.yml" + workflow_dispatch: + inputs: + dirs: + description: "Space-separated folders to audit, for example: examples/stock-notifications catalog/employeeRecognition" + required: true + +permissions: + contents: read + +concurrency: + group: audit-${{ github.event.pull_request.number || github.run_id }} + cancel-in-progress: true + +jobs: + audit: + runs-on: ubuntu-latest + env: + AUDIT_MODE: ${{ contains(github.event.pull_request.labels.*.name, 'audit-override') && 'advisory' || vars.AUDIT_MODE || 'enforcing' }} + BASE_SHA: ${{ github.event.pull_request.base.sha }} + HEAD_SHA: ${{ github.event.pull_request.head.sha }} + PR_NUMBER: ${{ github.event.pull_request.number }} + steps: + - uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - uses: actions/setup-node@v7 + with: + node-version: 20 + + - name: Find changed example folders + id: dirs + run: | + if [ -n "$PR_NUMBER" ]; then + node scripts/audit-examples.mjs --list-changed "$BASE_SHA" "$HEAD_SHA" + else + echo "dirs=${{ inputs.dirs }}" >> "$GITHUB_OUTPUT" + fi + + # Arcane's own GitHub Action installs the pinned CLI (sha256 verified) + # and reviews each folder. Annotations and the fail-on policy are left + # to the merge step below so hub rules and Arcane rules render together. + - name: Run Arcane Auditor + if: steps.dirs.outputs.dirs != '' + uses: Ekwuno/ArcaneAuditor@c31316d1147cb0e2d3f47688e668f7f3b2f9e887 # v2.0.0 CLI, action from the Ekwuno fork main + with: + path: ${{ steps.dirs.outputs.dirs }} + config: .arcane-auditor/config.json + output: audit/arcane.json + annotate: "false" + fail-on: none + + - name: Merge with hub rules and report + if: steps.dirs.outputs.dirs != '' + run: | + merge="" + [ -f audit/arcane.json ] && merge="--merge audit/arcane.json" + if [ -n "$PR_NUMBER" ]; then + node scripts/audit-examples.mjs --changed "$BASE_SHA" "$HEAD_SHA" $merge \ + --mode "$AUDIT_MODE" --format ci --output audit/report.json --pr "$PR_NUMBER" + else + node scripts/audit-examples.mjs --dirs ${{ steps.dirs.outputs.dirs }} $merge \ + --mode "$AUDIT_MODE" --format ci --output audit/report.json + fi + + - name: Nothing to audit + if: steps.dirs.outputs.dirs == '' + run: echo "No example folders changed." >> "$GITHUB_STEP_SUMMARY" + + # The comment workflow reads this artifact. Uploaded even when the + # check fails so contributors still get the suggestions. + - uses: actions/upload-artifact@v4 + if: always() && github.event_name == 'pull_request' && steps.dirs.outputs.dirs != '' + with: + name: audit-report + path: audit/ + retention-days: 7 diff --git a/.gitignore b/.gitignore index 4e67d6a..dd61be2 100644 --- a/.gitignore +++ b/.gitignore @@ -22,3 +22,7 @@ venv/ .DS_Store Thumbs.db *.log + +# Arcane Auditor local install and audit output +.arcane-auditor/bin/ +/audit/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0537a6d..df1cea6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -36,7 +36,7 @@ Thanks for helping build the open home for Workday Build examples. Adding an exa Each creates `examples/your-example-name/` with a prefilled `example.json` and README skeleton. (You can also copy `examples/_template/` by hand.) 3. **Drop your artifact in.** Whatever it is: Extend app source (exported with Local Disk Sync, the WDCLI, or the ZIP download), orchestration definitions, an agent skill as markdown, diagrams. The folder must be self-contained. -4. **Fill in the two files.** `example.json` needs a title, a description, and a type; everything else is optional. The README needs three short sections: What it is, What's inside, How to use it. +4. **Fill in the two files.** `example.json` needs a title, a description, and a type; everything else is optional. The README needs four short sections: What it is, What's inside, How to use it, and Before you deploy (everything a reader must change for their own tenant, such as app reference ids, base URLs, WIDs, or dates). 5. **Validate.** From the repository root: ```bash @@ -45,14 +45,23 @@ Thanks for helping build the open home for Workday Build examples. Adding an exa This checks your metadata and updates the README index table. Commit the README change with your example; CI runs the same script with `--check`. If you cannot run Node locally, skip this step and see Submitting by hand below. -6. **Open a pull request** and complete the short checklist in the PR template. +6. **Audit (optional but recommended).** The same checks CI runs on your pull request, with the fix for each finding: + + ```bash + ./scripts/install-arcane.sh + node scripts/audit-examples.mjs --changed + ``` + + The first command downloads [Arcane Auditor](https://github.com/Developers-and-Dragons/ArcaneAuditor), a community code review tool for Extend apps, into a gitignored folder. The second audits every example folder you changed. Without Arcane, `node scripts/audit-examples.mjs --changed --skip-arcane` still runs the hub's own checks (folder name, metadata, README sections, hardcoded values). What each finding means and how to fix it is in [docs/EXAMPLE_BEST_PRACTICES.md](docs/EXAMPLE_BEST_PRACTICES.md). + +7. **Open a pull request** and complete the short checklist in the PR template. ## Submitting by hand (no tooling required) The scaffolder and validator are conveniences, not requirements. The actual contract is just a folder under `examples/` containing your artifact plus `example.json` and `README.md`. To submit without running anything: 1. Copy `examples/_template/` into a new kebab-case folder, or create the files directly in the GitHub web UI in your fork. -2. Fill in `example.json` (the template's README documents every field) and write the three README sections. +2. Fill in `example.json` (the template's README documents every field) and write the four README sections. 3. Add your artifact files to the folder. 4. For the index table in the repository README, either add your row between the `` and `` markers by copying the format of an existing row, or leave the table alone and say so in your PR. CI will flag the stale table, and a reviewer will regenerate it for you during review. That is normal and fine. @@ -62,6 +71,7 @@ The scaffolder and validator are conveniences, not requirements. The actual cont - The README says what the artifact is and how to use it (deploy, import, read, or run). - `example.json` is valid: `type` comes from the `types` list in `hub.config.json`, and any `components` or `products` come from their lists too. - No credentials, tenant names, or real personal data anywhere in the folder. Sample data must be clearly fictional. +- Nothing tenant-specific is hardcoded without a note. Hardcoded Workday API URLs, app reference ids, and debug logging fail the audit; anything else a reader must change goes in the README under "Before you deploy". The full list of checks, with the fix for each, is in [docs/EXAMPLE_BEST_PRACTICES.md](docs/EXAMPLE_BEST_PRACTICES.md). ## Examples must be real Workday use cases @@ -98,6 +108,11 @@ Workday DevRel reviews every pull request before merge. We look for: - **It teaches.** The README explains the why, not just the how. - **It is safe.** No secrets, no real data, nothing tenant-specific. +Two automated checks run first and post their results on the pull request: + +- **Validate examples** checks `example.json` and the README index table. +- **Audit examples** runs Arcane Auditor plus the hub's own rules on the folders you changed, then comments on the PR with what to fix and how. Where the fix is mechanical you get a one-click suggestion. ACTION findings (hardcoded Workday URLs or app ids, debug logging, missing error handling) fail the check; ADVICE findings are recommendations and never block. If a finding is wrong for your example, say so in the PR and a maintainer can override it. + We aim to respond within a few business days. Discussions on the PR are part of the process, so expect questions and suggestions rather than a silent merge or close. Merged examples are labeled in the gallery: **Workday** for examples authored by Workday teams, **Community** for everything else. Community examples are held to works, safe, and honest; Workday-authored ones get a stricter pass because people copy them as reference. diff --git a/README.md b/README.md index 27353dc..3a96591 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,7 @@ scripts/ new-example.mjs Scaffold a new example folder in one command new-example.sh / .ps1 The same scaffolder for machines without Node validate-examples.mjs CI validation + README index generation + audit-examples.mjs Arcane Auditor + hub rules, same audit CI runs on PRs site/ Optional Astro gallery (not required to use the examples) hub.config.json Repo URLs and the approved type, component, and product lists ``` @@ -124,7 +125,7 @@ We want your examples. Community contributions go into `examples/` (the Examples 1. Scaffold a folder: `node scripts/new-example.mjs your-example-name`. No Node? `./scripts/new-example.sh` (macOS, Linux) and `scripts\new-example.ps1` (Windows) do the same thing. 2. Drop your artifact in, and fill in the generated `example.json` and README. -3. Validate: `node scripts/validate-examples.mjs` +3. Validate: `node scripts/validate-examples.mjs`, then audit: `./scripts/install-arcane.sh && node scripts/audit-examples.mjs --changed` (optional; CI runs it on your PR and explains every finding in [docs/EXAMPLE_BEST_PRACTICES.md](docs/EXAMPLE_BEST_PRACTICES.md)) 4. Open a pull request. Workday DevRel reviews every submission before merge. Prefer to do it by hand? No tooling is required. Copy [`examples/_template`](examples/_template) into a new folder (you can even create the files straight from the GitHub web UI), fill in the two files, and open the PR. The index table above can be edited by hand, or a reviewer will regenerate it for you during review. diff --git a/docs/EXAMPLE_BEST_PRACTICES.md b/docs/EXAMPLE_BEST_PRACTICES.md new file mode 100644 index 0000000..9ff247f --- /dev/null +++ b/docs/EXAMPLE_BEST_PRACTICES.md @@ -0,0 +1,384 @@ +# Example best practices + +Every example in this hub exists to be copied into someone else's tenant. This page lists the things that make that copy succeed or fail, and it is the reference the automated audit links to. Each heading is a rule id; when the **Audit examples** check flags something on your pull request, the link in that finding lands on the matching section here. + +The audit combines two sources: + +- [Arcane Auditor](https://github.com/Developers-and-Dragons/ArcaneAuditor), a community code review tool for Workday Extend and Orchestrate source by Chris Humphrey (MIT). Its rule ids look like `HardcodedWorkdayAPIRule`. The explanations below are adapted from its rule documentation; run `ArcaneAuditorCLI describe-rule ` for the full version. +- The hub's own rules in `scripts/audit/hub-rules.mjs`, about packaging and portability rather than code. Their ids start with `Hub`. + +Two severities. **ACTION** findings fail the check and should be fixed before merge. **ADVICE** findings are suggestions and never block. A maintainer can add the `audit-override` label when a finding is wrong for a particular example. Examples in `catalog/` are held to a stricter bar (ADVICE counts as ACTION there) because people copy them as reference. + +The review rubric behind all of this is the one in [CONTRIBUTING.md](../CONTRIBUTING.md): it works, it teaches, it is safe. + +## Contents + +- [The "Before you deploy" contract](#the-before-you-deploy-contract) +- [Portability](#portability): `HardcodedWorkdayAPIRule`, `HardcodedApplicationIdRule`, `HardcodedWidRule`, `EndpointBaseUrlTypeRule`, `HubAppReferenceIdRule`, `HubHardcodedPeriodLiteralRule` +- [Robustness and safety](#robustness-and-safety): endpoints, security domains, orchestration error handling +- [Clean code](#clean-code): debug logging, `var`, dead code, string building, structure +- [Naming and layout](#naming-and-layout) +- [Hub packaging](#hub-packaging): folder name, `example.json`, README sections, leftover template text, `.gitkeep` +- [Orchestrate](#orchestrate) +- [Messages from the tooling itself](#messages-from-the-tooling-itself) +- [Running the audit yourself](#running-the-audit-yourself) + +## The "Before you deploy" contract + +The single most useful thing an example README can do is tell the reader what to change. A hardcoded value is not a problem when the README says "replace this". It becomes a problem when the reader only finds out at runtime. + +Every example README has a `## Before you deploy` section (the template in `examples/_template/README.md` has a skeleton). List each tenant-specific thing by file and value: + +```markdown +## Before you deploy + +- **App reference id**: `presentation/promotionNomination_rvylxm.amd` and `.smd` carry the id from the + original tenant. Rename the files and replace `promotionNomination_rvylxm` with your own id. +- **Promotion cycle**: `managerNomination.pmd` sets `promotionCycleWidget` to `2026-Q1`. Change it, or + replace it with a script that derives the quarter from today's date. +- **Worker type WID**: the WQL in `getEmployeeList` filters on a WID from the original tenant. + Replace it with the id of the worker type you want. +- **Security domains**: map `ManagerPromotionNomination` to your manager security group. +``` + +Several rules below relax when this section exists and mentions the value in question. Documentation that says "in order to use this, update xyz" mitigates most concerns a reviewer has. + +## Portability + +### HardcodedWorkdayAPIRule + +**ACTION, one-click fix.** A URL like `https://api.workday.com/common/v1/workers/me` or `https://api.us.wcp.workday.com/wql/v1` is pinned to one region and one infrastructure generation. Anyone who imports the example in another data center gets failing calls, and Workday cannot move the endpoint under you. + +Fix: use the `apiGatewayEndpoint` application variable, or better, a data provider plus `baseUrlType` (see [EndpointBaseUrlTypeRule](#endpointbaseurltyperule)). + +```json +// before +{ "name": "workerInfo", "url": "https://api.workday.com/common/v1/workers/me" } + +// after +{ "name": "workerInfo", "baseUrlType": "workday-common", "url": "/workers/me" } +``` + +When the URL sits inside a script expression, swap the literal for the variable rather than nesting a second `<% %>`: + +```json +"url": "<% apiGatewayEndpoint + '/businessProcess/v1/events/' + queryParams.eventId %>" +``` + +### HardcodedApplicationIdRule + +**ACTION in this hub (Arcane's default is ADVICE).** The application id Workday generates (`promotionNomination_rvylxm`) is unique per tenant. Baking it into a URL or a data provider guarantees the copied example breaks on import. The fix is mechanical, which is why the hub raises it to ACTION. + +```json +// before +"url": "<% 'https://api.workday.com/apps/promotionNomination_rvylxm/v1/nominations' %>" + +// after +"url": "<% apiGatewayEndpoint + '/apps/' + site.applicationId + '/v1/nominations' %>" + +// or, with a data provider in the .amd and baseUrlType "app" +"baseUrlType": "app", "url": "/nominations" +``` + +Arcane only detects this when the `.smd` is present in the folder, because that is where it reads the id from. Keep the `.smd` in the example. + +### HardcodedWidRule + +**ADVICE.** A 32-character Workday id (`d588c41a446c11de98360015c5e6daf6`) usually belongs to one tenant. Even when it happens to exist everywhere, nobody reading the code knows what it points at. + +Fix: store it in an app attribute (`attributes/default.attributes`) with a meaningful name and read it as `appAttr.`. Tenant admins set the value per tenant. If the example needs a literal for teaching purposes, name it in the README's "Before you deploy" section. + +### EndpointBaseUrlTypeRule + +**ADVICE.** Endpoints that spell out a Workday URL, even through `apiGatewayEndpoint`, duplicate the same host across every page. Define the host once as a data provider in the `.amd` and use `baseUrlType` on each endpoint. + +```json +// .amd +"dataProviders": [ { "key": "workday-common", "value": "<% apiGatewayEndpoint + '/common/v1' %>" } ] + +// .pmd +{ "name": "me", "baseUrlType": "workday-common", "url": "/workers/me" } +``` + +Declaring `baseUrlType` and an absolute URL on the same endpoint is a sign the second one won the copy-paste. Pick one. + +### HubAppReferenceIdRule + +**ADVICE.** The `.amd` and `.smd` file names and their `applicationId` / `siteId` fields carry the tenant-generated id (`stocknotifications_svfbfp`). That is normal for an export. What matters is that the reader knows to replace it. + +Fix: add a `## Before you deploy` section to the README that names the id and says to replace it. The rule does not fire when that section exists. Scripts should reference the id as `site.applicationId` rather than repeating the literal. + +### HubHardcodedPeriodLiteralRule + +**ADVICE.** A widget whose `value` is a period or date literal (`2026-Q1`, `FY2026`, `2026-03`) is correct for exactly one cycle. After that someone has to edit and redeploy the app, and an example that is silently wrong teaches the wrong lesson. + +Fix, in order of preference: + +1. Derive it. A few lines in the `script` section can turn `date:today` into a quarter label. +2. Read it from an app attribute so admins change it without a deploy. +3. Keep the literal and mention it, with the file name, under "Before you deploy". The rule does not fire when the README mentions the literal there. + +Legitimate defaults for a date picker demo are fine; document them. + +## Robustness and safety + +### EndpointFailOnStatusCodesRule + +**ACTION.** Without `failOnStatusCodes`, a 400 or 403 from the API does not fail the endpoint. The page carries on as if the call succeeded and shows empty or stale data with no error. Add at least 400 and 403 to every endpoint: + +```json +{ "name": "getWorkers", "url": "/workers", "failOnStatusCodes": [ { "code": 400 }, { "code": 403 } ] } +``` + +### OnlyMaximumEffortRule + +**ACTION.** `"bestEffort": true` tells Extend to ignore failures on that endpoint. In an example that hides exactly the errors a reader needs to see. Remove it. + +### NoIsCollectionOnEndpointsRule + +**ACTION.** `"isCollection": true` on an inbound endpoint pulls entire collections and has caused tenant-wide slowdowns under concurrent use. Use a WQL or RaaS query that returns what the page needs. + +### NoPMDSessionVariablesRule + +**ACTION.** An `outboundVariable` with `"variableScope": "session"` lives for the whole login session and keeps consuming memory after the user leaves the page. Use `"variableScope": "flow"`. + +### PMDSecurityDomainRule + +**ACTION.** A page with no `securityDomains` is open to every user in the tenant. Add at least one, and describe in "Before you deploy" which security group the reader should map it to. MicroConclusion pages and error pages listed in the `.smd` are exempt. + +### WidgetIdRequiredRule + +**ACTION.** Widgets without an `id` cannot be referenced from scripts, are hard to find in error messages, and in some widget types (panelList, for example) do not log their values. Give every section, fieldSet, richText, and input an id. Built-in containers like `footer`, `item`, `group`, `title`, `pod`, and `card` are exempt. + +### GridPagingWithSortableFilterableRule + +**ACTION.** Paging combined with `sortableAndFilterable` columns re-fetches, re-sorts, and re-filters on every page change. Choose one: drop paging, or turn off sorting and filtering on the columns. + +### StringBooleanRule + +**ADVICE, one-click fix.** `"enabled": "false"` is a string that happens to be cast. `"enabled": false` says what you mean. Some AMD flow fields do require strings; Arcane skips those. + +### MultipleStringInterpolatorsRule + +**ADVICE, one-click fix.** `"<% a %> and <% b %>"` in one string is harder to read than a single template: `` "<% `{{a}} and {{b}}` %>" ``. + +## Clean code + +### ScriptConsoleLogRule + +**ACTION, one-click fix.** `console.info`, `console.debug`, `console.warn`, and `console.error` calls in a shipped page write to tenant logs that other people can read, and in an example they teach readers to ship debug output. Remove them. The audit suggests commenting the call out so you can decide; deleting it is better. If an example needs logging, gate it behind an app attribute and say so. + +Commented-out console calls do not trigger the rule. + +### ScriptVarUsageRule + +**ADVICE, one-click fix.** `var` is function-scoped, so reusing a name in a nested block overwrites the outer value. Use `let` for values that change and `const` for values that do not. + +### ScriptStringConcatRule + +**ADVICE, one-click fix where the expression fits on one line.** `'Hello ' + name + '!'` is easy to get wrong (missing spaces, wrong types). Extend's template syntax handles it: `` `Hello {{name}}!` ``. + +### ScriptVerboseBooleanCheckRule + +**ADVICE, one-click fix.** `if (isActive == true)` and `return cond ? true : false` restate a boolean. Write `if (isActive)` and `return cond`. + +### ScriptUnusedVariableRule + +**ADVICE.** A variable that is declared and never read is usually a leftover from a refactor. Remove it so readers do not go looking for where it is used. + +### ScriptUnusedFunctionRule + +**ADVICE.** A function in a PMD or Pod `script` section that is never called from that file is dead weight. Remove it, or move it to a `.script` file if another page needs it. + +### ScriptUnusedFunctionParametersRule + +**ADVICE.** Parameters the function body never touches make callers guess what to pass. Drop them. + +### ScriptUnusedIncludesRule + +**ADVICE, one-click fix.** An `include` entry for a `.script` file whose functions are never called (`script.function()`) still costs parse time on every page load. Remove it. + +### ScriptDeadCodeRule + +**ADVICE.** In a standalone `.script` file, every top-level declaration should either be exported in the final object literal or used by something that is. Anything else is loaded for nothing. + +### ScriptEmptyFunctionRule + +**ADVICE.** An empty function body is either unfinished or unnecessary. Implement it or delete it. + +### ScriptMagicNumberRule + +**ADVICE.** `if (score > 85)` hides what 85 means. Name it: `const promotionThreshold = 85`. In a teaching example the name is the lesson. + +### ScriptComplexityRule + +**ADVICE.** More than ten independent paths through one function (every `if`, loop, and `&&` adds one) is hard to test and hard to read. Split it into smaller functions with descriptive names. + +### ScriptNestingLevelRule + +**ADVICE.** More than four nested `if` / `for` levels is hard to follow. Flatten with early returns or extracted functions. + +### ScriptLongFunctionRule + +**ADVICE.** Functions over 50 lines are doing several things. Split them. + +### ScriptLongBlockRule + +**ADVICE.** Inline handlers (`onLoad`, `onChange`, `onSend`) over 30 lines belong in named functions in the `script` section, or in a `.script` file if shared. + +### ScriptFunctionParameterCountRule + +**ADVICE.** More than four parameters invites wrong-order bugs. Pass an object, or split the function. + +### ScriptFunctionReturnConsistencyRule + +**ADVICE.** If some paths return a value and others return nothing, callers get `null` by surprise. Make every path return explicitly, `null` included. + +### ScriptArrayMethodUsageRule + +**ADVICE.** Manual index loops are where off-by-one bugs live. `map`, `filter`, and `forEach` say what the loop is for. + +### ScriptNestedArraySearchRule + +**ADVICE.** `workers.map(w => orgs.find(o => o.id == w.orgId))` searches the whole inner array for every outer item. With thousands of records that is slow and can run out of memory. Build a map once with `list:toMap()` and look up by key. + +### ScriptOnSendSelfDataRule + +**ADVICE.** Assigning a new object to `self.data` inside an outbound endpoint's `onSend` uses the endpoint as a scratch variable. Build the payload in a local variable and return it. Setting properties on existing `self.data` (from `valueOutBinding`) is fine. + +### ScriptDescriptiveParameterRule + +**ADVICE.** `users.filter(x => x.active)` reads better as `users.filter(user => user.active)`. `a` and `b` in sort comparators are fine. + +### EmbeddedImagesRule + +**ADVICE.** A base64 `data:image/...` value makes the file large, makes every diff huge, and is not cacheable. Reference the image by URL, or ship it as a separate file in the example folder and say so in the README. + +## Naming and layout + +### FileNameLowerCamelCaseRule + +**ADVICE.** Extend file names are lowerCamelCase: `managerNomination.pmd`, not `Manager_Nomination.pmd`. Exported `.amd` and `.smd` files carry the generated app id, which contains an underscore; that is expected and covered by [HubAppReferenceIdRule](#hubappreferenceidrule) instead. + +### EndpointNameLowerCamelCaseRule + +**ADVICE.** Endpoint names are lowerCamelCase: `getEmployeeList`, not `GetEmployeeList` or `get_employee_list`. + +### WidgetIdLowerCamelCaseRule + +**ADVICE.** Widget ids are lowerCamelCase for the same reason. + +### ScriptVariableNamingRule + +**ADVICE.** Variables are lowerCamelCase. + +### ScriptFunctionParameterNamingRule + +**ADVICE.** Parameters are lowerCamelCase. + +### PMDSectionOrderingRule + +**ADVICE.** Top-level PMD sections in a consistent order (`id`, `securityDomains`, `include`, `script`, `endPoints`, `onSubmit`, `outboundData`, `onLoad`, `presentation`) make every page scan the same way. Reorder the keys; nothing else changes. + +### FooterPodRequiredRule + +**ADVICE.** A footer defined inline on every page has to be edited on every page. Put it in a pod and include the pod. Hub, tabbed, and microConclusion pages are exempt. + +## Hub packaging + +These rules come from the hub, not Arcane. They are about the folder as a unit of reuse. + +### HubFolderKebabCaseRule + +**ACTION.** Example folders are kebab-case: `promotion-nomination`, not `Promotion_Nomination` or `PromotionNomination`. The folder name becomes the URL slug in the gallery and the id in the README index, and the scaffolder enforces the same rule. The finding tells you the name to use. + +### HubExampleJsonRule + +**ACTION.** The folder needs `example.json` and `README.md`. That is the whole contract; the gallery, the index tables, and the badges are built from it. `example.json` needs `title`, `description`, and a `type` from `hub.config.json`; `components` and `products` also come from that file. The template's README documents every field. + +Common findings and their fixes: + +- *example.json is missing, "app-info.json" looks like the metadata file*: rename the file. +- *starts with "JSON" before the JSON begins*: a pasted code-fence label. Delete the first line. +- *"X" is not an approved type*: pick one of the values listed in the message. + +The **Validate examples** check reports the same problems from `scripts/validate-examples.mjs`. + +### HubReadmeSectionsRule + +**ACTION for the first three, ADVICE for the fourth.** The README has four markdown sections, in this order: + +1. `## What it is`: what the example shows and who it is for. +2. `## What's inside`: the files in the folder and what each is. +3. `## How to use it`: deploy, import, read, or run. +4. `## Before you deploy`: what to change for another tenant. See [the contract](#the-before-you-deploy-contract). + +A README written in raw HTML (`

`) gets one finding asking for the markdown sections. HTML does not render the same way in the gallery, and the sections are what reviewers and readers scan for. + +### HubTemplateBoilerplateRule + +**ACTION.** Text from `examples/_template/` is still in the submission: "One short paragraph: what this example shows", the "Fill in example.json (delete this section before submitting)" block, or the title `My Example`. Replace it with your content and delete the instructions block. + +### HubGitkeepRule + +**ADVICE.** `.gitkeep` exists only to make git keep an empty folder. Once the folder has real files the placeholder is noise. Delete it. + +## Orchestrate + +### OrchestrationSecurityDomainRule + +**ACTION.** Sync and Async orchestrations must declare a security domain, otherwise anyone can invoke them. Add one and name the intended security group in "Before you deploy". + +### OrchestrationGlobalErrorHandlerRule + +**ADVICE in this hub (Arcane's default is ACTION).** A global error handler with a Log step (or Add Integration Message for integration templates) records failures that no local handler caught. Worth adding to any orchestration people will copy; not always the point of a minimal example, which is why the hub downgrades it. + +### OrchestrationApiStepErrorHandlerRule + +**ADVICE in this hub (Arcane's default is ACTION).** Each API step should have a local error handler with a Log step, so transient failures are recorded rather than swallowed. Same reasoning as above. + +### OrchestrationBranchOnConditionsNestingRule + +**ADVICE.** More than three nested Branch on Conditions steps is hard to follow in the builder. Extract a suborchestration. + +### OrchestratePreferExplicitDefaultAccessor + +**ADVICE.** Some accessor functions throw when a value is missing. Prefer the variant that takes a default so the missing case is handled where it happens. + +### OrchestrationVerboseBooleanCheckRule + +**ADVICE.** A Conditional wrapper that returns `true` or `false` around a condition that is already boolean is redundant. Use the condition directly. + +## Messages from the tooling itself + +### ArcaneAuditorError + +**ACTION.** Arcane Auditor could not analyze the folder at all (it exited with a usage or runtime error). The message carries the first line of the error. Usually a file it could not read; check the folder builds in your own tooling and ask in the PR if it is not obvious. This is never about your code style. + +### ArcaneAuditorWarning + +**ADVICE.** Arcane analyzed the folder but its script parser gave up on one block, so script rules were skipped for that block. The message names the line. Often the script uses a construct Arcane's grammar does not know yet, which is worth an issue on the Arcane repository; sometimes it is a real syntax slip worth a second look. + +## Running the audit yourself + +From the repository root: + +```bash +./scripts/install-arcane.sh # once; downloads the Arcane CLI into .arcane-auditor/bin/ +node scripts/audit-examples.mjs --changed +``` + +`--changed` audits every example folder that differs from `origin/main`. Other forms: + +```bash +node scripts/audit-examples.mjs --dirs examples/my-example # one folder +node scripts/audit-examples.mjs --changed --skip-arcane # hub rules only, no download needed +node scripts/audit-examples.mjs --changed --format markdown # what the PR comment will say +node scripts/audit-examples.mjs --changed --format json --output audit/report.json +``` + +The exit code is 1 when there are blocking findings, the same as CI. + +**Reading the PR comment.** The audit posts one summary comment (updated on every push, never duplicated) and inline review comments on lines it can point at. Where the fix is a one-line substitution the inline comment carries a GitHub suggestion you can apply with one click. Findings on lines the PR did not change, and file-level findings, appear only in the summary. + +**Configuration.** Arcane's rule settings for this hub live in `.arcane-auditor/config.json`; the reasons for each deviation from Arcane's defaults are in `.arcane-auditor/README.md`. To propose a change to a rule, a severity, or this page, open an issue or a pull request touching those files. Changes to Arcane's rules themselves belong upstream in the Arcane Auditor repository. diff --git a/examples/README.md b/examples/README.md index 86b8359..e18b035 100644 --- a/examples/README.md +++ b/examples/README.md @@ -9,3 +9,5 @@ Add your own in minutes: - No tooling at all? Copy [_template/](_template/) by hand, even from the GitHub web UI Fill in the two generated files, drop your artifact in, and open a pull request. The [contributing guide](../CONTRIBUTING.md) has the details, and Workday DevRel reviews every submission before merge. + +Every pull request is audited automatically for the things that make an example hard to reuse: hardcoded Workday URLs and app ids, debug logging, missing error handling, a README without deploy notes. The audit comments on the PR with the fix for each finding. To run it yourself first: `./scripts/install-arcane.sh && node scripts/audit-examples.mjs --changed`. The checks are explained in [docs/EXAMPLE_BEST_PRACTICES.md](../docs/EXAMPLE_BEST_PRACTICES.md). diff --git a/examples/_template/README.md b/examples/_template/README.md index 0cec834..748d2ea 100644 --- a/examples/_template/README.md +++ b/examples/_template/README.md @@ -23,6 +23,16 @@ The concrete steps to put this example to work, whatever that means for this art If your example needs configuration (credentials, tenant URLs), document the variables here and never commit real values. +## Before you deploy + +Everything a reader has to change before this example works in their own tenant. Delete the lines that do not apply. Typical items: + +- **App reference id**: `presentation/myApp_abc123.amd` and `.smd` carry the id Workday generated for the original tenant. Replace `myApp_abc123` with your own, or reference it with `site.applicationId` in scripts. +- **Base URLs**: endpoints use `baseUrlType` with the data providers in the `.amd`; check they match your region. +- **Security domains**: `SampleDomain` in `model/` needs to be mapped to your security groups. +- **Dates or periods**: `2026-Q1` in `managerNomination.pmd` is the current cycle; change it or compute it. +- **WIDs**: any Workday id in a WQL query is from the original tenant and must be replaced. + --- ## Fill in example.json (delete this section before submitting) diff --git a/scripts/audit-examples.mjs b/scripts/audit-examples.mjs new file mode 100755 index 0000000..3a1579a --- /dev/null +++ b/scripts/audit-examples.mjs @@ -0,0 +1,178 @@ +#!/usr/bin/env node +// Audits example folders with Arcane Auditor plus the hub's own rules, and +// reports what to fix and how. Zero dependencies, Node 20. +// +// node scripts/audit-examples.mjs --changed folders changed vs origin/main +// node scripts/audit-examples.mjs --changed folders changed in a range (CI) +// node scripts/audit-examples.mjs --dirs examples/foo ... specific folders +// node scripts/audit-examples.mjs --all every folder (slow, noisy) +// +// Options +// --mode enforcing|advisory enforcing (default) exits 1 on blocking findings +// --format console|json|markdown|ci ci = annotations + job summary + JSON file +// --output write the JSON report here (default audit/report.json for ci) +// --pr record the PR number in the report and audit/pr.json +// --merge use a report from the Arcane GitHub Action instead of running it +// --skip-arcane hub rules only (no binary needed) +// --hub-only | --arcane-only run one side +// --rules A,B --exclude-rules A,B forwarded to Arcane +// --list-changed print changed folders and exit (sets dirs= in $GITHUB_OUTPUT) +// +// Exit codes: 0 clean or advisory, 1 blocking findings, 2 usage, 3 tooling failure. + +import { existsSync, mkdirSync, readdirSync, statSync, writeFileSync, appendFileSync } from "node:fs"; +import { dirname, join, resolve } from "node:path"; +import { repoRoot, sections } from "./validate-examples.mjs"; +import { changedDirs, diffLineMap, dirInfo } from "./audit/diff.mjs"; +import { listRules, loadArcaneReport, runArcane } from "./audit/arcane.mjs"; +import { HUB_RULES, runHubRules } from "./audit/hub-rules.mjs"; +import { buildReport } from "./audit/report.mjs"; +import { toAnnotations, toConsole, toMarkdown } from "./audit/render.mjs"; + +const CONFIG_PATH = ".arcane-auditor/config.json"; + +const args = parseArgs(process.argv.slice(2)); +if (args.help) { + console.log(usage()); + process.exit(0); +} + +// Which folders --------------------------------------------------------------- +let dirs = []; +let diffMap = null; +if (args["list-changed"] || args.changed) { + const [base, head] = args["list-changed"] ?? args.changed; + try { + dirs = changedDirs(base, head); + diffMap = diffLineMap(base, head); + } catch (err) { + fail(3, `git diff failed: ${err.message}\nFetch the base branch first (git fetch origin main) or pass --dirs.`); + } + if (args["list-changed"]) { + const list = dirs.map((d) => d.path).join(" "); + console.log(list); + if (process.env.GITHUB_OUTPUT) appendFileSync(process.env.GITHUB_OUTPUT, `dirs=${list}\n`); + process.exit(0); + } +} else if (args.dirs) { + dirs = args.dirs.map((d) => { + const info = dirInfo(d); + if (!sections.some((s) => s.dir === info.section) || !existsSync(join(repoRoot, info.path))) fail(2, `Not an example folder: ${d}`); + return info; + }); +} else if (args.all) { + for (const s of sections) { + const abs = join(repoRoot, s.dir); + if (!existsSync(abs)) continue; + for (const name of readdirSync(abs).sort()) { + if (name.startsWith("_") || name.startsWith(".") || !statSync(join(abs, name)).isDirectory()) continue; + dirs.push({ path: `${s.dir}/${name}`, section: s.dir, name, status: "modified" }); + } + } +} else { + fail(2, usage()); +} + +const mode = args.mode ?? "enforcing"; +if (!["enforcing", "advisory"].includes(mode)) fail(2, `--mode must be enforcing or advisory (got ${mode})`); +const format = args.format ?? "console"; +if (!["console", "json", "markdown", "ci"].includes(format)) fail(2, `--format must be console, json, markdown, or ci (got ${format})`); + +// Arcane ---------------------------------------------------------------------- +let arcaneReport = null; +if (!args["skip-arcane"] && !args["hub-only"] && dirs.length > 0) { + if (args.merge) { + try { + arcaneReport = loadArcaneReport(resolve(repoRoot, args.merge)); + } catch (err) { + fail(3, `Could not load Arcane report ${args.merge}: ${err.message}`); + } + } else { + try { + arcaneReport = runArcane(dirs.map((d) => d.path), { configPath: existsSync(join(repoRoot, CONFIG_PATH)) ? CONFIG_PATH : undefined, rules: args.rules, excludeRules: args["exclude-rules"] }); + } catch (err) { + fail(3, err.message); + } + } +} + +// Hub rules ------------------------------------------------------------------- +let hubFindings = []; +if (!args["arcane-only"]) { + for (const d of dirs) hubFindings.push(...runHubRules(d)); +} + +// Report ---------------------------------------------------------------------- +const pr = args.pr ? { number: Number(args.pr), base: args.changed?.[0] ?? null, head: args.changed?.[1] ?? null } : null; +const report = buildReport({ dirs, arcaneReport, hubFindings, mode, diffMap, pr, rulesMeta: arcaneReport ? listRules() : new Map(), hubRulesMeta: HUB_RULES }); + +const output = args.output ?? (format === "ci" ? "audit/report.json" : null); +if (output) { + const outPath = resolve(repoRoot, output); + mkdirSync(dirname(outPath), { recursive: true }); + writeFileSync(outPath, JSON.stringify(report, null, 2) + "\n"); + if (pr) writeFileSync(join(dirname(outPath), "pr.json"), JSON.stringify(pr) + "\n"); +} + +switch (format) { + case "json": + if (!output) console.log(JSON.stringify(report, null, 2)); + break; + case "markdown": + console.log(toMarkdown(report)); + break; + case "ci": + for (const line of toAnnotations(report)) console.log(line); + if (process.env.GITHUB_STEP_SUMMARY) appendFileSync(process.env.GITHUB_STEP_SUMMARY, toMarkdown(report)); + console.log(`\n${report.summary.effective_action} blocking, ${report.summary.effective_advice} advisory finding(s) in ${dirs.length} folder(s). Mode: ${mode}. Report: ${output}`); + break; + default: + console.log(toConsole(report)); +} + +process.exit(mode === "enforcing" && report.summary.effective_action > 0 ? 1 : 0); + +// --------------------------------------------------------------------------- + +function parseArgs(argv) { + const out = {}; + const listFlags = new Set(["dirs"]); + const rangeFlags = new Set(["changed", "list-changed"]); + const valueFlags = new Set(["mode", "format", "output", "pr", "merge", "rules", "exclude-rules"]); + for (let i = 0; i < argv.length; i++) { + const a = argv[i]; + if (!a.startsWith("--")) fail(2, `Unexpected argument: ${a}\n\n${usage()}`); + const key = a.slice(2); + if (key === "help" || key === "h") out.help = true; + else if (listFlags.has(key)) { + out[key] = []; + while (argv[i + 1] && !argv[i + 1].startsWith("--")) out[key].push(argv[++i].replace(/\/+$/, "")); + } else if (rangeFlags.has(key)) { + out[key] = []; + while (argv[i + 1] && !argv[i + 1].startsWith("--") && out[key].length < 2) out[key].push(argv[++i]); + } else if (valueFlags.has(key)) { + if (!argv[i + 1] || argv[i + 1].startsWith("--")) fail(2, `--${key} needs a value`); + out[key] = argv[++i]; + } else if (["all", "skip-arcane", "hub-only", "arcane-only"].includes(key)) out[key] = true; + else fail(2, `Unknown option --${key}\n\n${usage()}`); + } + return out; +} + +function usage() { + return `Usage: + node scripts/audit-examples.mjs --changed [ []] + node scripts/audit-examples.mjs --dirs [ ...] + node scripts/audit-examples.mjs --all + node scripts/audit-examples.mjs --list-changed + +Options: + --mode enforcing|advisory --format console|json|markdown|ci --output + --pr --merge --skip-arcane --hub-only --arcane-only + --rules A,B --exclude-rules A,B`; +} + +function fail(code, msg) { + console.error(msg); + process.exit(code); +} diff --git a/scripts/audit/arcane.mjs b/scripts/audit/arcane.mjs new file mode 100644 index 0000000..3c145db --- /dev/null +++ b/scripts/audit/arcane.mjs @@ -0,0 +1,203 @@ +// Runs Arcane Auditor locally, or loads a report produced by the Arcane +// GitHub Action in CI. Zero dependencies. +// +// Report shape (Arcane schema 2.0 plus what the action's merge adds): +// { schema_version, summary, runs: [{path, status, exit_code, findings}], findings: [...] } +// findings[].location.file_path is repo-relative (prefixed with the audited dir). + +import { spawnSync, execFileSync } from "node:child_process"; +import { existsSync, readFileSync, readdirSync, statSync } from "node:fs"; +import { homedir } from "node:os"; +import { join } from "node:path"; +import { repoRoot } from "../validate-examples.mjs"; + +export const ARCANE_EXTENSIONS = [".pod", ".pmd", ".script", ".amd", ".smd", ".wqlquery", ".orchestration", ".suborchestration"]; + +// Returns { cmd: [..argv prefix..] } or null when no binary is available. +export function findArcane() { + if (process.env.ARCANE_AUDITOR_CMD) return { cmd: process.env.ARCANE_AUDITOR_CMD.split(/\s+/).filter(Boolean) }; + const candidates = [ + process.env.ARCANE_AUDITOR_BIN, + join(repoRoot, ".arcane-auditor", "bin", "ArcaneAuditorCLI"), + join(homedir(), ".arcane-auditor", "bin", "ArcaneAuditorCLI") + ].filter(Boolean); + for (const c of candidates) if (existsSync(c)) return { cmd: [c] }; + try { + const which = execFileSync(process.platform === "win32" ? "where" : "which", ["ArcaneAuditorCLI"], { encoding: "utf8", stdio: ["ignore", "pipe", "ignore"] }).trim(); + if (which) return { cmd: [which.split("\n")[0]] }; + } catch { + // not on PATH + } + return null; +} + +export function hasArcaneFiles(dir) { + const abs = join(repoRoot, dir); + const stack = [abs]; + while (stack.length) { + const d = stack.pop(); + for (const name of readdirSync(d)) { + const p = join(d, name); + if (statSync(p).isDirectory()) stack.push(p); + else if (ARCANE_EXTENSIONS.some((ext) => name.toLowerCase().endsWith(ext))) return true; + } + } + return false; +} + +// Runs review-app for each dir and merges the results, mirroring the +// GitHub Action's merge step so local and CI reports look the same. +export function runArcane(dirs, { configPath, rules, excludeRules } = {}) { + const arcane = findArcane(); + if (!arcane) throw new Error("Arcane Auditor not found. Run ./scripts/install-arcane.sh, or set ARCANE_AUDITOR_BIN, or use --skip-arcane."); + + const runs = []; + const findings = []; + let totalFiles = 0; + let totalRules = 0; + + for (const dir of dirs) { + const run = { path: dir, status: "ok", exit_code: null }; + if (!existsSync(join(repoRoot, dir))) { + run.status = "missing"; + runs.push(run); + continue; + } + if (!hasArcaneFiles(dir)) { + run.status = "skipped"; + runs.push(run); + continue; + } + const args = [...arcane.cmd.slice(1), "review-app", dir, "--agent"]; + if (configPath) args.push("--config", configPath); + if (rules) args.push("--rules", rules); + if (excludeRules) args.push("--exclude-rules", excludeRules); + const res = spawnSync(arcane.cmd[0], args, { cwd: repoRoot, encoding: "utf8", maxBuffer: 64 * 1024 * 1024 }); + run.exit_code = res.status; + if (res.error || res.status >= 2 || res.status === null) { + run.status = "error"; + run.stderr = tail(res.stderr || String(res.error || ""), 20); + findings.push(errorFinding(dir, res.status ?? 3, res.stderr || String(res.error || ""))); + runs.push(run); + continue; + } + let report; + let preamble = ""; + try { + ({ report, preamble } = parseAgentJson(res.stdout)); + } catch (err) { + run.status = "error"; + run.stderr = `could not parse JSON output: ${err.message}`; + findings.push(errorFinding(dir, 3, run.stderr)); + runs.push(run); + continue; + } + for (const f of report.findings ?? []) { + const loc = f.location ?? {}; + const fp = loc.file_path && !loc.file_path.startsWith(dir + "/") ? `${dir}/${loc.file_path}` : loc.file_path; + findings.push({ ...f, location: { ...loc, file_path: fp } }); + } + if (preamble.trim()) { + run.warnings = tail(preamble, 12); + findings.push(warningFinding(dir, preamble)); + } + totalFiles += report.summary?.total_files ?? 0; + totalRules = Math.max(totalRules, report.summary?.total_rules ?? 0); + run.findings = (report.findings ?? []).length; + runs.push(run); + } + + const bySeverity = { ACTION: 0, ADVICE: 0 }; + for (const f of findings) bySeverity[f.severity] = (bySeverity[f.severity] ?? 0) + 1; + return { + schema_version: "2.0", + generated_by: "audit-examples.mjs", + summary: { total_files: totalFiles, total_rules: totalRules, total_findings: findings.length, findings_by_severity: bySeverity }, + runs, + findings + }; +} + +export function loadArcaneReport(path) { + const { report } = parseAgentJson(readFileSync(path, "utf8")); + if (!Array.isArray(report.findings)) throw new Error(`${path} is not an Arcane report (no findings array)`); + // The GitHub Action records parser warnings on the run; surface them the + // same way a local run does. + for (const run of report.runs ?? []) { + if (run.warnings && !report.findings.some((f) => f.rule_id === "ArcaneAuditorWarning" && f.location?.file_path === run.path)) { + report.findings.push(warningFinding(run.path, run.warnings)); + } + } + return report; +} + +// rule_id -> { description, severity, fix_strategy, category }. Empty when +// the binary is unavailable; callers must cope. +let rulesCache = null; +export function listRules() { + if (rulesCache) return rulesCache; + rulesCache = new Map(); + const arcane = findArcane(); + if (!arcane) return rulesCache; + const res = spawnSync(arcane.cmd[0], [...arcane.cmd.slice(1), "list-rules", "--format", "json"], { cwd: repoRoot, encoding: "utf8" }); + if (res.status !== 0) return rulesCache; + try { + for (const r of JSON.parse(res.stdout)) rulesCache.set(r.rule_id, r); + } catch { + // ignore + } + return rulesCache; +} + +// Agent mode prints JSON to stdout, but the CLI can still print a warning +// line first (for example about a config it had to normalize). Skip anything +// before the first line that starts the JSON document. +export function parseAgentJson(stdout) { + const text = String(stdout); + const idx = text.search(/^[{\[]/m); + if (idx === -1) throw new Error(`no JSON document in output: ${text.trim().split("\n")[0] ?? ""}`); + return { report: JSON.parse(text.slice(idx)), preamble: text.slice(0, idx) }; +} + +// Arcane printed a warning before the JSON, usually that its script parser +// gave up on one block. Script rules were skipped for that block, which is +// worth telling the contributor. +function warningFinding(dir, preamble) { + const first = (preamble.trim().split("\n")[0] ?? "").replace(/^Warning:\s*/i, "").trim(); + return { + rule_id: "ArcaneAuditorWarning", + severity: "ADVICE", + category: "tooling", + fix_strategy: "human_review", + fix_strategy_overridden: false, + message: `Arcane Auditor could not parse part of this folder, so some script rules were skipped: ${first}`, + location: { file_path: dir, line: 0, column: null, end_line: null, end_column: null, path: null }, + snippet: tail(preamble, 8), + suggested_replacement: null, + target_text: null, + replacement_context: null, + finding_id: `warning:${dir}` + }; +} + +function errorFinding(dir, code, stderr) { + const first = (stderr.trim().split("\n")[0] ?? "").trim(); + return { + rule_id: "ArcaneAuditorError", + severity: "ACTION", + category: "tooling", + fix_strategy: "human_review", + fix_strategy_overridden: false, + message: code === 2 ? `Arcane Auditor could not analyze this folder (usage error). ${first}` : `Arcane Auditor failed while analyzing this folder (exit ${code}). ${first}`, + location: { file_path: dir, line: 0, column: null, end_line: null, end_column: null, path: null }, + snippet: tail(stderr, 10), + suggested_replacement: null, + target_text: null, + replacement_context: null, + finding_id: `error:${dir}:${code}` + }; +} + +function tail(text, n) { + return String(text).trim().split("\n").slice(-n).join("\n"); +} diff --git a/scripts/audit/diff.mjs b/scripts/audit/diff.mjs new file mode 100644 index 0000000..1ce8719 --- /dev/null +++ b/scripts/audit/diff.mjs @@ -0,0 +1,101 @@ +// Git helpers for the example audit. Zero dependencies. +// +// changedDirs(base, head) -> entry folders under catalog/ or examples/ touched +// between base and head, with added|modified status +// diffLineMap(base, head) -> Map> of right-side lines in the diff, +// used to tell new code from pre-existing code + +import { execFileSync } from "node:child_process"; +import { existsSync, statSync } from "node:fs"; +import { join } from "node:path"; +import { repoRoot, sections } from "../validate-examples.mjs"; + +const sectionDirs = sections.map((s) => s.dir); + +export function git(args, opts = {}) { + return execFileSync("git", args, { cwd: repoRoot, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"], ...opts }); +} + +export function resolveRange(base, head) { + base = base || "origin/main"; + head = head || "HEAD"; + // Three-dot range: changes on head since it diverged from base. + return `${base}...${head}`; +} + +export function changedDirs(base, head) { + const range = resolveRange(base, head); + const names = git(["diff", "--name-only", "--diff-filter=ACMRD", range, "--", ...sectionDirs]) + .split("\n") + .filter(Boolean); + + const seen = new Map(); + for (const file of names) { + const parts = file.split("/"); + // Only files inside an entry folder count: examples//... . A file + // sitting directly under examples/ (its README) is not an entry. + if (parts.length < 3) continue; + const [section, name] = parts; + if (!section || !name || !sectionDirs.includes(section)) continue; + if (name.startsWith("_") || name.startsWith(".")) continue; + const path = `${section}/${name}`; + if (seen.has(path)) continue; + // Skip folders deleted in this range. Check the working tree first (CI + // checks out the PR head) and fall back to the head commit for local runs. + const abs = join(repoRoot, path); + const inTree = existsSync(abs) && statSync(abs).isDirectory(); + if (!inTree && !existsAt(head || "HEAD", path)) continue; + seen.set(path, { path, section, name, status: existedAt(base || "origin/main", head || "HEAD", path) ? "modified" : "added" }); + } + return [...seen.values()]; +} + +function existsAt(ref, path) { + try { + return git(["ls-tree", "-d", "--name-only", ref, "--", path]).trim() !== ""; + } catch { + return false; + } +} + +function existedAt(base, head, path) { + // The merge base is what the PR started from. + let ref = base; + try { + ref = git(["merge-base", base, head]).trim(); + } catch { + // fall back to base as given + } + try { + return git(["ls-tree", "-d", "--name-only", ref, "--", path]).trim() !== ""; + } catch { + return false; + } +} + +export function diffLineMap(base, head) { + const range = resolveRange(base, head); + const out = git(["diff", "-U0", "--no-color", range, "--", ...sectionDirs]); + const map = new Map(); + let file = null; + for (const line of out.split("\n")) { + if (line.startsWith("+++ ")) { + file = line.startsWith("+++ b/") ? line.slice(6) : null; + continue; + } + const m = line.match(/^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@/); + if (m && file) { + const start = Number(m[1]); + const count = m[2] === undefined ? 1 : Number(m[2]); + if (!map.has(file)) map.set(file, new Set()); + const set = map.get(file); + for (let i = 0; i < count; i++) set.add(start + i); + } + } + return map; +} + +export function dirInfo(path) { + const [section, name] = path.replace(/\/+$/, "").split("/"); + return { path: `${section}/${name}`, section, name, status: "modified" }; +} diff --git a/scripts/audit/hub-rules.mjs b/scripts/audit/hub-rules.mjs new file mode 100644 index 0000000..b534ae0 --- /dev/null +++ b/scripts/audit/hub-rules.mjs @@ -0,0 +1,237 @@ +// Hub-specific rules that Arcane Auditor does not cover: folder naming, the +// example.json contract, README sections, leftover template text, stray +// .gitkeep files, hardcoded period literals, and tenant-specific app ids. +// Zero dependencies. +// +// Every rule returns findings shaped like: +// { rule_id, severity, fix_strategy, message, file, line, +// target_text?, suggested_replacement?, replacement_context? } +// file is repo-relative; line is 0 for file-level findings. + +import { existsSync, readFileSync, readdirSync, statSync } from "node:fs"; +import { join, relative } from "node:path"; +import { repoRoot, validateEntry } from "../validate-examples.mjs"; + +export const HUB_RULES = new Map([ + ["HubFolderKebabCaseRule", { severity: "ACTION", description: "Example folders are kebab-case so URLs, gallery cards, and the scaffolder agree." }], + ["HubExampleJsonRule", { severity: "ACTION", description: "Every example has a valid example.json and README.md, the contract the gallery and index are built from." }], + ["HubReadmeSectionsRule", { severity: "ACTION", description: "The README has the sections readers rely on: What it is, What's inside, How to use it, and Before you deploy." }], + ["HubTemplateBoilerplateRule", { severity: "ACTION", description: "Placeholder text from the template was left in the submission." }], + ["HubGitkeepRule", { severity: "ADVICE", description: ".gitkeep files only exist to keep empty folders; delete them once the folder has content." }], + ["HubHardcodedPeriodLiteralRule", { severity: "ADVICE", description: "A date or period literal (like 2026-Q1) is hardcoded, so the example silently goes stale." }], + ["HubAppReferenceIdRule", { severity: "ADVICE", description: "A tenant-generated app reference id (the _xxxxxx suffix) is baked into the example without telling readers to replace it." }] +]); + +const KEBAB = /^[a-z0-9][a-z0-9-]*$/; // same rule as scripts/new-example.mjs +const REQUIRED_SECTIONS = [ + { key: "what it is", label: "What it is", severity: "ACTION" }, + { key: "what's inside", label: "What's inside", severity: "ACTION", alt: /what.?s inside|what is inside|contents/ }, + { key: "how to use it", label: "How to use it", severity: "ACTION", alt: /how to use|usage|getting started|setup|how to run|deploy/ }, + { key: "before you deploy", label: "Before you deploy", severity: "ADVICE", alt: /before you deploy|before deploying|what to change|what you need to change|customi[sz]e for your tenant/ } +]; +const TEMPLATE_STRINGS = [ + "Fill in example.json (delete this section before submitting)", + "One short paragraph: what this example shows and who it is for", + "Bullet the contents of this folder so a reader knows", + "The concrete steps to put this example to work, whatever that means", + "One or two sentences about what this example shows." +]; +const PERIOD_LITERAL = /^(?:\d{4}-(?:Q[1-4]|H[12]|\d{2})(?:-\d{2})?|FY\d{2,4}|\d{4})$/; +const APP_REF_ID = /^[A-Za-z0-9]+_[a-z]{6}$/; + +// dir: { path, section, name } +export function runHubRules(dir) { + const ctx = context(dir); + return [ + ...folderKebabCase(ctx), + ...exampleJson(ctx), + ...readmeSections(ctx), + ...templateBoilerplate(ctx), + ...gitkeep(ctx), + ...hardcodedPeriodLiteral(ctx), + ...appReferenceId(ctx) + ]; +} + +function context(dir) { + const abs = join(repoRoot, dir.path); + const readmePath = join(abs, "README.md"); + const readme = existsSync(readmePath) ? readFileSync(readmePath, "utf8") : null; + const headings = readme ? [...readme.matchAll(/^#{1,3}\s+(.+?)\s*#*\s*$/gm)].map((m) => m[1].trim()) : []; + const files = existsSync(abs) ? walk(abs).map((p) => relative(repoRoot, p).replace(/\\/g, "/")) : []; + const hasBeforeDeploy = headings.some((h) => REQUIRED_SECTIONS[3].alt.test(h.toLowerCase())); + return { ...dir, abs, readme, headings, files, hasBeforeDeploy }; +} + +function folderKebabCase(ctx) { + if (KEBAB.test(ctx.name)) return []; + const suggested = ctx.name + .replace(/([a-z0-9])([A-Z])/g, "$1-$2") + .replace(/[^a-zA-Z0-9]+/g, "-") + .toLowerCase() + .replace(/^-+|-+$/g, ""); + return [finding("HubFolderKebabCaseRule", ctx.path, 0, `Folder name "${ctx.name}" is not kebab-case (lowercase letters, digits, and hyphens). Rename it to "${suggested}".`, { suggested_replacement: suggested, target_text: ctx.name, replacement_context: "rename" })]; +} + +function exampleJson(ctx) { + const out = []; + const { errors } = validateEntry(ctx.section, ctx.name); + const rootJson = ctx.files.filter((f) => f.split("/").length === 3 && f.endsWith(".json")); + for (const err of errors) { + const msg = err.replace(`${ctx.path}: `, "").replace(`${ctx.path}/`, ""); + if (msg === "missing README.md") { + out.push(finding("HubExampleJsonRule", `${ctx.path}/README.md`, 0, "README.md is missing. Copy examples/_template/README.md and fill in its sections.")); + continue; + } + if (msg === "missing example.json") { + const candidate = rootJson.find((f) => looksLikeMeta(f)); + const hint = candidate + ? ` "${candidate.split("/").pop()}" looks like the metadata file; rename it to example.json.` + : " Copy examples/_template/example.json and fill in title, description, and type."; + out.push(finding("HubExampleJsonRule", `${ctx.path}/example.json`, 0, `example.json is missing.${hint}`, candidate ? { target_text: candidate.split("/").pop(), suggested_replacement: "example.json", replacement_context: "rename" } : {})); + continue; + } + out.push(finding("HubExampleJsonRule", `${ctx.path}/example.json`, 0, msg.startsWith("example.json") ? msg : `example.json: ${msg}`)); + } + // A JSON file whose first line is not JSON (a pasted code-fence label, say). + for (const f of rootJson) { + const text = read(f); + const first = text.split(/\r?\n/)[0] ?? ""; + if (first.trim() && !/^[\s{\[]/.test(first)) { + out.push(finding("HubExampleJsonRule", f, 1, `${f.split("/").pop()} starts with "${first.trim()}" before the JSON begins. Delete that line so the file parses.`, { target_text: first, suggested_replacement: "", replacement_context: "full_line" })); + } + } + return out; +} + +function looksLikeMeta(f) { + try { + const text = read(f).replace(/^[^{\[]*/, ""); + const j = JSON.parse(text); + return typeof j.title === "string" && typeof j.description === "string"; + } catch { + return /"title"\s*:/.test(read(f)) && /"description"\s*:/.test(read(f)); + } +} + +function readmeSections(ctx) { + if (ctx.readme == null) return []; // reported by exampleJson + const lower = ctx.headings.map((h) => h.toLowerCase()); + const missing = REQUIRED_SECTIONS.filter((s) => !lower.some((h) => h.includes(s.key) || (s.alt && s.alt.test(h)))); + if (missing.length === 0) return []; + const rawHtml = ctx.headings.length === 0 && /]/i.test(ctx.readme); + const out = []; + if (rawHtml) { + out.push(finding("HubReadmeSectionsRule", `${ctx.path}/README.md`, 0, `README.md uses HTML headings instead of markdown. Rewrite it with the four markdown sections from examples/_template/README.md: ${REQUIRED_SECTIONS.map((s) => `"## ${s.label}"`).join(", ")}.`)); + return out; + } + const blocking = missing.filter((s) => s.severity === "ACTION"); + const advice = missing.filter((s) => s.severity === "ADVICE"); + if (blocking.length) { + out.push(finding("HubReadmeSectionsRule", `${ctx.path}/README.md`, 0, `README.md is missing ${blocking.length === 1 ? "the section" : "the sections"} ${blocking.map((s) => `"## ${s.label}"`).join(", ")}. Readers need to know what the example is, what is in the folder, and how to use it.`)); + } + for (const s of advice) { + out.push({ ...finding("HubReadmeSectionsRule", `${ctx.path}/README.md`, 0, `README.md has no "## ${s.label}" section. List everything a reader must change before this works in their tenant: app reference ids, base URLs, WIDs, security domains, dates or periods.`), severity: "ADVICE" }); + } + return out; +} + +function templateBoilerplate(ctx) { + const out = []; + for (const f of ctx.files.filter((p) => p.endsWith("README.md") || p.endsWith("example.json"))) { + const text = read(f); + for (const needle of TEMPLATE_STRINGS) { + const idx = text.indexOf(needle); + if (idx === -1) continue; + out.push(finding("HubTemplateBoilerplateRule", f, lineAt(text, idx), `Template placeholder text is still here: "${needle.slice(0, 60)}...". Replace it with your own content.`)); + } + if (f.endsWith("example.json") && /"title"\s*:\s*"My Example"/.test(text)) { + out.push(finding("HubTemplateBoilerplateRule", f, lineAt(text, text.indexOf('"My Example"')), 'example.json still has the template title "My Example". Give the example a real title.')); + } + } + return out; +} + +function gitkeep(ctx) { + const out = []; + for (const f of ctx.files.filter((p) => p.endsWith("/.gitkeep"))) { + const folder = join(repoRoot, f, ".."); + const others = readdirSync(folder).filter((n) => n !== ".gitkeep"); + if (others.length === 0) continue; + out.push({ ...finding("HubGitkeepRule", f, 0, `.gitkeep is no longer needed because "${relative(repoRoot, folder).replace(/\\/g, "/")}" has ${others.length} other file(s). Delete it.`, { replacement_context: "file_remove" }), severity: "ADVICE", fix_strategy: "actionable" }); + } + return out; +} + +function hardcodedPeriodLiteral(ctx) { + const out = []; + for (const f of ctx.files.filter((p) => /\.(pmd|pod)$/i.test(p))) { + const text = read(f); + for (const m of text.matchAll(/"value"\s*:\s*"([^"<]{4,12})"/g)) { + const literal = m[1]; + if (!PERIOD_LITERAL.test(literal)) continue; + const mentioned = ctx.hasBeforeDeploy && ctx.readme.includes(literal); + if (mentioned) continue; + const line = lineAt(text, m.index); + out.push({ + ...finding("HubHardcodedPeriodLiteralRule", f, line, `"${literal}" is a hardcoded period or date. Every cycle someone has to edit and redeploy the app. Compute it from today's date (for example \`<% date:today %>\` and a small script), read it from an app attribute, or document it under "## Before you deploy" so readers know to change it.`), + severity: "ADVICE" + }); + } + } + return out; +} + +function appReferenceId(ctx) { + if (ctx.hasBeforeDeploy) return []; + const out = []; + const seen = new Set(); + for (const f of ctx.files.filter((p) => /\.(amd|smd)$/i.test(p))) { + const base = f.split("/").pop().replace(/\.(amd|smd)$/i, ""); + const text = read(f); + const ids = new Set(); + if (APP_REF_ID.test(base)) ids.add(base); + for (const m of text.matchAll(/"(?:applicationId|siteId|id)"\s*:\s*"([A-Za-z0-9]+_[a-z]{6})"/g)) ids.add(m[1]); + for (const id of ids) { + const key = `${f}:${id}`; + if (seen.has(key)) continue; + seen.add(key); + const idx = text.indexOf(`"${id}"`); + out.push({ + ...finding("HubAppReferenceIdRule", f, idx >= 0 ? lineAt(text, idx) : 0, `"${id}" is the app reference id Workday generated for the original tenant (the _${id.split("_").pop()} suffix). Anyone who imports this example gets a different suffix. Add a "## Before you deploy" section to the README that tells readers to replace it, or reference it dynamically with site.applicationId in scripts.`), + severity: "ADVICE" + }); + } + } + return out; +} + +// helpers ------------------------------------------------------------------- + +function finding(rule_id, file, line, message, extra = {}) { + const meta = HUB_RULES.get(rule_id); + return { rule_id, severity: meta.severity, fix_strategy: extra.suggested_replacement != null ? "actionable" : "human_review", message, file, line, ...extra }; +} + +const cache = new Map(); +function read(f) { + if (!cache.has(f)) cache.set(f, readFileSync(join(repoRoot, f), "utf8")); + return cache.get(f); +} + +function lineAt(text, idx) { + let n = 1; + for (let i = 0; i < idx && i < text.length; i++) if (text.charCodeAt(i) === 10) n++; + return n; +} + +function walk(dir) { + const out = []; + for (const name of readdirSync(dir)) { + if (name === "node_modules" || name === ".git") continue; + const p = join(dir, name); + if (statSync(p).isDirectory()) out.push(...walk(p)); + else out.push(p); + } + return out; +} diff --git a/scripts/audit/post-review.mjs b/scripts/audit/post-review.mjs new file mode 100644 index 0000000..bb9aaa5 --- /dev/null +++ b/scripts/audit/post-review.mjs @@ -0,0 +1,206 @@ +// Posts a hub-audit/1 report to a pull request as a sticky summary comment +// and inline review comments with one-click suggestions. Runs inside +// actions/github-script in .github/workflows/audit-comment.yml with a write +// token, so it treats the report strictly as data: every field is validated +// and capped, the PR is resolved from the trusted workflow_run event, and +// nothing from the report is ever executed. + +import { readFileSync } from "node:fs"; +import { suggestionFor } from "./report.mjs"; +import { toMarkdown } from "./render.mjs"; + +const STICKY_MARKER = ""; +const MAX_FINDINGS = 500; +const MAX_INLINE = 40; +const PATH_RE = /^(examples|catalog)\/[^\0]+$/; + +export function validateReport(raw) { + if (!raw || typeof raw !== "object") throw new Error("report is not an object"); + if (raw.schema_version !== "hub-audit/1") throw new Error(`unexpected schema_version ${raw.schema_version}`); + if (!["enforcing", "advisory"].includes(raw.mode)) throw new Error("bad mode"); + if (!Array.isArray(raw.findings) || raw.findings.length > MAX_FINDINGS) throw new Error("findings missing or too many"); + if (!Array.isArray(raw.dirs)) throw new Error("dirs missing"); + const dirs = raw.dirs.map((d) => { + if (typeof d.path !== "string" || !PATH_RE.test(d.path) || d.path.includes("..")) throw new Error(`bad dir path ${d.path}`); + return { path: d.path, section: d.section === "catalog" ? "catalog" : "examples", name: str(d.name, 200), status: d.status === "added" ? "added" : "modified" }; + }); + const findings = raw.findings.map((f) => { + if (typeof f.file !== "string" || !PATH_RE.test(f.file) || f.file.includes("..")) throw new Error(`bad file path ${f.file}`); + const line = Number.isInteger(f.line) && f.line >= 0 ? f.line : 0; + return { + id: str(f.id, 300), + source: f.source === "hub" ? "hub" : "arcane", + rule_id: str(f.rule_id, 80).replace(/[^A-Za-z0-9_]/g, ""), + severity: f.severity === "ACTION" ? "ACTION" : "ADVICE", + effective_severity: f.effective_severity === "ACTION" ? "ACTION" : "ADVICE", + downgrade_reason: f.downgrade_reason ? str(f.downgrade_reason, 40) : null, + promote_reason: f.promote_reason ? str(f.promote_reason, 40) : null, + fix_strategy: f.fix_strategy === "actionable" ? "actionable" : "human_review", + message: str(f.message, 1000), + why: f.why ? str(f.why, 500) : null, + file: f.file, + dir: str(f.dir, 300), + line, + target_text: f.target_text != null ? str(f.target_text, 500) : null, + suggested_replacement: f.suggested_replacement != null ? str(f.suggested_replacement, 2000) : null, + replacement_context: f.replacement_context != null ? str(f.replacement_context, 30) : null, + source_line: f.source_line != null ? str(f.source_line, 2000) : null, + in_diff: f.in_diff === true ? true : f.in_diff === false ? false : null, + doc_url: typeof f.doc_url === "string" && /^https:\/\/github\.com\//.test(f.doc_url) ? str(f.doc_url, 300) : null + }; + }); + const s = raw.summary ?? {}; + return { + schema_version: raw.schema_version, + mode: raw.mode, + dirs, + findings, + summary: { action: num(s.action), advice: num(s.advice), effective_action: num(s.effective_action), effective_advice: num(s.effective_advice) } + }; +} + +// The PR comes from the workflow_run event, never from the artifact. +// workflow_run.pull_requests is empty for forks, so look it up by head. +export async function resolvePr({ github, context, headSha, headBranch, headOwner }) { + const { data } = await github.rest.pulls.list({ ...context.repo, state: "open", head: `${headOwner}:${headBranch}`, per_page: 10 }); + const pr = data.find((p) => p.head.sha === headSha); + return pr ?? null; +} + +// Map> of right-side lines a review comment may attach to. +export async function commentableLines({ github, context, prNumber }) { + const map = new Map(); + const files = await github.paginate(github.rest.pulls.listFiles, { ...context.repo, pull_number: prNumber, per_page: 100 }); + for (const f of files) { + if (!f.patch) continue; + const set = new Set(); + for (const line of f.patch.split("\n")) { + const m = line.match(/^@@ -\d+(?:,\d+)? \+(\d+)(?:,(\d+))? @@/); + if (!m) continue; + const start = Number(m[1]); + const count = m[2] === undefined ? 1 : Number(m[2]); + for (let i = 0; i < count; i++) set.add(start + i); + } + map.set(f.filename, set); + } + return map; +} + +export function buildReview(report, lineMap, alreadyPosted = new Set()) { + const comments = []; + const overflow = []; + for (const f of report.findings) { + if (alreadyPosted.has(f.id)) continue; + const canInline = f.line > 0 && lineMap.get(f.file)?.has(f.line); + if (!canInline) { + overflow.push(f); + continue; + } + if (comments.length >= MAX_INLINE) { + overflow.push(f); + continue; + } + comments.push({ path: f.file, line: f.line, side: "RIGHT", body: inlineBody(f) }); + } + return { comments, overflow }; +} + +function inlineBody(f) { + const parts = []; + parts.push(`**${f.rule_id}** (${f.effective_severity}${f.effective_severity !== f.severity ? `, was ${f.severity}` : ""})`); + parts.push(""); + parts.push(f.message); + if (f.why) { + parts.push(""); + parts.push(`Why: ${f.why}`); + } + const suggestion = suggestionFor(f); + if (suggestion != null) { + parts.push(""); + parts.push("```suggestion"); + parts.push(suggestion); + parts.push("```"); + } else if (f.suggested_replacement) { + parts.push(""); + parts.push("Suggested change:"); + parts.push("```"); + parts.push(f.suggested_replacement); + parts.push("```"); + } + if (f.doc_url) { + parts.push(""); + parts.push(`[Read more](${f.doc_url})`); + } + parts.push(""); + parts.push(``); + return parts.join("\n"); +} + +export async function existingFindingIds({ github, context, prNumber }) { + const ids = new Set(); + const comments = await github.paginate(github.rest.pulls.listReviewComments, { ...context.repo, pull_number: prNumber, per_page: 100 }); + for (const c of comments) { + if (!c.user || c.user.type !== "Bot") continue; + const m = c.body?.match(//); + if (m) ids.add(m[1].trim()); + } + return ids; +} + +export async function upsertStickyComment({ github, context, prNumber, body }) { + const comments = await github.paginate(github.rest.issues.listComments, { ...context.repo, issue_number: prNumber, per_page: 100 }); + const existing = comments.find((c) => c.body?.includes(STICKY_MARKER) && c.user?.type === "Bot"); + if (existing) { + await github.rest.issues.updateComment({ ...context.repo, comment_id: existing.id, body }); + } else { + await github.rest.issues.createComment({ ...context.repo, issue_number: prNumber, body }); + } +} + +export async function postReview({ github, context, core, reportPath, headSha, headBranch, headOwner, checkConclusion }) { + let report; + try { + report = validateReport(JSON.parse(readFileSync(reportPath, "utf8"))); + } catch (err) { + core.warning(`Audit report rejected: ${err.message}`); + return; + } + const pr = await resolvePr({ github, context, headSha, headBranch, headOwner }); + if (!pr) { + core.info(`No open pull request found for ${headOwner}:${headBranch} at ${headSha}; nothing to post.`); + return; + } + const prNumber = pr.number; + + const lineMap = await commentableLines({ github, context, prNumber }); + const posted = await existingFindingIds({ github, context, prNumber }); + const { comments, overflow } = buildReview(report, lineMap, posted); + + if (comments.length > 0) { + await github.rest.pulls.createReview({ + ...context.repo, + pull_number: prNumber, + commit_id: headSha, + event: "COMMENT", + body: `Example audit: ${comments.length} inline suggestion(s). The summary comment on this PR has the full list.`, + comments + }); + core.info(`Posted ${comments.length} inline comment(s).`); + } + + let body = toMarkdown(report, { forComment: true }); + if (checkConclusion === "failure" && report.mode === "enforcing") { + body = body.replace("\n", "\n> The **Audit examples** check failed because of the items under *Fix before merge*. Push a fix and it re-runs automatically.\n\n"); + } + const notInline = overflow.filter((f) => !posted.has(f.id) && f.line === 0).length; + if (notInline > 0 && comments.length > 0) body += `\n${notInline} finding(s) are file-level and appear only in this summary.\n`; + await upsertStickyComment({ github, context, prNumber, body }); + core.info(`Updated summary comment on #${prNumber}.`); +} + +function str(v, max) { + return String(v ?? "").slice(0, max); +} +function num(v) { + return Number.isInteger(v) && v >= 0 ? v : 0; +} diff --git a/scripts/audit/render.mjs b/scripts/audit/render.mjs new file mode 100644 index 0000000..3041242 --- /dev/null +++ b/scripts/audit/render.mjs @@ -0,0 +1,145 @@ +// Renders a hub-audit/1 report for the console, GitHub annotations, the job +// summary, and the sticky PR comment. Zero dependencies. + +import { suggestionFor } from "./report.mjs"; + +const LOCAL_HELP = "Run it yourself: `./scripts/install-arcane.sh` once, then `node scripts/audit-examples.mjs --changed`. Rule explanations and fixes: [docs/EXAMPLE_BEST_PRACTICES.md](" + docRoot() + ")."; + +function docRoot() { + return "docs/EXAMPLE_BEST_PRACTICES.md"; +} + +export function toConsole(report) { + const out = []; + const s = report.summary; + if (report.findings.length === 0) { + out.push(`Audit: no findings in ${report.dirs.map((d) => d.path).join(", ") || "no folders"}.`); + return out.join("\n"); + } + let lastFile = null; + for (const f of report.findings) { + if (f.file !== lastFile) { + out.push(""); + out.push(f.file); + lastFile = f.file; + } + const tag = f.effective_severity === f.severity ? f.effective_severity : `${f.effective_severity} (was ${f.severity}: ${f.downgrade_reason ?? f.promote_reason})`; + out.push(` ${f.line > 0 ? `line ${f.line}` : "file"} ${tag} ${f.rule_id}`); + out.push(` ${f.message}`); + const sug = suggestionFor(f); + if (sug != null) { + out.push(` fix: ${f.target_text ? `replace "${f.target_text}" with "${f.suggested_replacement}"` : sug}`); + } else if (f.suggested_replacement) { + out.push(` suggested: ${f.suggested_replacement}`); + } + out.push(` docs: ${f.doc_url}`); + } + out.push(""); + out.push(`${s.effective_action} blocking, ${s.effective_advice} advisory (${s.action} ACTION / ${s.advice} ADVICE before policy). Mode: ${report.mode}.`); + return out.join("\n"); +} + +// GitHub workflow commands. At most 10 errors and 10 warnings render per step. +export function toAnnotations(report, maxPerLevel = 10) { + const lines = []; + const count = { error: 0, warning: 0 }; + for (const f of report.findings) { + const level = f.effective_severity === "ACTION" ? "error" : "warning"; + if (count[level] >= maxPerLevel) continue; + count[level]++; + const props = [`file=${escProp(f.file)}`]; + if (f.line > 0) props.push(`line=${f.line}`); + props.push(`title=${escProp(`${f.rule_id} (${f.effective_severity})`)}`); + let msg = f.message; + if (f.target_text && f.suggested_replacement) msg += ` Suggested fix: replace "${f.target_text}" with "${f.suggested_replacement}".`; + msg += ` See ${f.doc_url}`; + lines.push(`::${level} ${props.join(",")}::${escData(msg)}`); + } + const hidden = report.findings.length - count.error - count.warning; + if (hidden > 0) lines.push(`::notice title=Example audit::${hidden} more finding(s) are listed in the job summary.`); + return lines; +} + +// Markdown for the job summary and the sticky PR comment. +export function toMarkdown(report, { title = "Example audit", forComment = false } = {}) { + const s = report.summary; + const out = []; + if (forComment) out.push(""); + out.push(`## ${title}`); + out.push(""); + const folders = report.dirs.map((d) => `\`${d.path}\``).join(", "); + if (report.dirs.length === 0) { + out.push("No example folders changed, nothing to audit."); + return out.join("\n") + "\n"; + } + if (report.findings.length === 0) { + out.push(`Audited ${folders}. No findings. Thank you for a clean submission.`); + out.push(""); + out.push(LOCAL_HELP); + return out.join("\n") + "\n"; + } + + const passed = []; + if (!report.findings.some((f) => f.rule_id.startsWith("Hub"))) passed.push("hub packaging checks (folder name, example.json, README sections)"); + if (!report.findings.some((f) => f.rule_id === "HardcodedWorkdayAPIRule" || f.rule_id === "HardcodedApplicationIdRule")) passed.push("no hardcoded Workday URLs or app ids"); + if (!report.findings.some((f) => f.rule_id === "ScriptConsoleLogRule")) passed.push("no debug logging"); + if (passed.length) out.push(`Audited ${folders}. Passed: ${passed.join("; ")}.`); + else out.push(`Audited ${folders}.`); + out.push(""); + + if (report.mode === "advisory") { + out.push("> Advisory mode: nothing here blocks the merge, but the ACTION items would in enforcing mode."); + out.push(""); + } + out.push(`**${s.effective_action} to fix** and **${s.effective_advice} suggestion(s)**.`); + out.push(""); + + const blocking = report.findings.filter((f) => f.effective_severity === "ACTION"); + const advisory = report.findings.filter((f) => f.effective_severity === "ADVICE"); + if (blocking.length) { + out.push("### Fix before merge"); + out.push(""); + out.push(...table(blocking)); + out.push(""); + } + if (advisory.length) { + out.push(`Suggestions (${advisory.length}, never block)`); + out.push(""); + out.push(...table(advisory)); + out.push(""); + out.push(""); + out.push(""); + } + out.push(LOCAL_HELP); + if (forComment) { + out.push(""); + out.push("ACTION items fail the **Audit examples** check. ADVICE never blocks. Maintainers can add the `audit-override` label to merge with open ACTION items."); + } + return out.join("\n") + "\n"; +} + +function table(findings) { + const rows = ["| Where | Rule | What to change |", "| --- | --- | --- |"]; + for (const f of findings.slice(0, 150)) { + const where = f.line > 0 ? `\`${f.file}\` line ${f.line}` : `\`${f.file}\``; + const rule = `[${f.rule_id}](${f.doc_url})`; + let what = f.message; + if (f.target_text && f.suggested_replacement) what += ` Replace \`${f.target_text}\` with \`${f.suggested_replacement}\`.`; + else if (f.suggested_replacement && !f.target_text) what += ` Suggested: \`${f.suggested_replacement}\`.`; + if (f.downgrade_reason === "pre-existing-line") what += " (pre-existing code, not blocking)"; + if (f.promote_reason === "catalog-strict") what += " (catalog apps are held to the stricter bar)"; + rows.push(`| ${where} | ${rule} | ${cell(what)} |`); + } + if (findings.length > 150) rows.push(`| | | ... and ${findings.length - 150} more |`); + return rows; +} + +function cell(s) { + return String(s).replace(/\|/g, "\\|").replace(/\r?\n/g, " "); +} +function escData(s) { + return String(s).replace(/%/g, "%25").replace(/\r/g, "%0D").replace(/\n/g, "%0A"); +} +function escProp(s) { + return escData(s).replace(/:/g, "%3A").replace(/,/g, "%2C"); +} diff --git a/scripts/audit/report.mjs b/scripts/audit/report.mjs new file mode 100644 index 0000000..629df57 --- /dev/null +++ b/scripts/audit/report.mjs @@ -0,0 +1,209 @@ +// Builds the unified audit report (schema hub-audit/1) from Arcane findings +// and hub-rule findings, and applies the severity policy. Zero dependencies. +// +// Policy, in order: +// 1. catalog/ folders are held to a stricter bar: ADVICE counts as ACTION. +// 2. In a folder that already existed, ACTION findings on lines the PR did +// not touch are downgraded to ADVICE so contributors are only blocked on +// what they wrote. +// 3. In advisory mode nothing blocks: everything becomes ADVICE. +// +// effective_severity is what CI acts on; severity is what the rule said. + +import { existsSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { config as hubConfig, repoRoot } from "../validate-examples.mjs"; + +export const SCHEMA = "hub-audit/1"; +export const DOC_PATH = "docs/EXAMPLE_BEST_PRACTICES.md"; + +export function docUrl(ruleId) { + return `${hubConfig.repoUrl}/blob/${hubConfig.defaultBranch}/${DOC_PATH}#${ruleId.toLowerCase()}`; +} + +// dirs: [{ path, section, name, status: added|modified }] +// arcaneReport: merged Arcane report or null +// hubFindings: findings from hub-rules.mjs (already repo-relative) +// mode: enforcing | advisory +// diffMap: Map> or null when no diff context exists +// rulesMeta: Map from Arcane list-rules (may be empty) +export function buildReport({ dirs, arcaneReport, hubFindings, mode, diffMap, pr, rulesMeta, hubRulesMeta }) { + const dirByPath = new Map(dirs.map((d) => [d.path, d])); + const findings = []; + + for (const f of arcaneReport?.findings ?? []) { + const loc = f.location ?? {}; + findings.push( + normalize({ + source: "arcane", + rule_id: f.rule_id, + severity: f.severity, + fix_strategy: f.fix_strategy ?? "human_review", + message: f.message ?? "", + why: rulesMeta?.get(f.rule_id)?.description ?? null, + file: loc.file_path ?? "", + line: loc.line ?? 0, + end_line: loc.end_line ?? null, + json_path: loc.path ?? null, + snippet: clip(f.snippet, 400), + target_text: f.target_text ?? null, + suggested_replacement: f.suggested_replacement ?? null, + replacement_context: f.replacement_context ?? null + }, dirByPath, diffMap) + ); + } + + for (const f of hubFindings ?? []) { + findings.push( + normalize({ + source: "hub", + why: hubRulesMeta?.get(f.rule_id)?.description ?? null, + end_line: null, + json_path: null, + snippet: null, + target_text: null, + suggested_replacement: null, + replacement_context: null, + ...f + }, dirByPath, diffMap) + ); + } + + for (const f of findings) applyPolicy(f, dirByPath.get(f.dir), mode, diffMap); + + findings.sort((a, b) => rank(a) - rank(b) || cmp(a.file, b.file) || cmp(a.line, b.line) || cmp(a.rule_id, b.rule_id) || cmp(a.message, b.message)); + + const summary = { action: 0, advice: 0, effective_action: 0, effective_advice: 0, by_rule: {}, by_dir: {} }; + for (const f of findings) { + summary[f.severity === "ACTION" ? "action" : "advice"]++; + summary[f.effective_severity === "ACTION" ? "effective_action" : "effective_advice"]++; + summary.by_rule[f.rule_id] = (summary.by_rule[f.rule_id] ?? 0) + 1; + summary.by_dir[f.dir] = (summary.by_dir[f.dir] ?? 0) + 1; + } + + return { + schema_version: SCHEMA, + mode, + generated_at: new Date().toISOString(), + pr: pr ?? null, + dirs: dirs.map((d) => ({ + ...d, + arcane: (arcaneReport?.runs ?? []).find((r) => r.path === d.path) ?? { status: arcaneReport ? "skipped" : "not-run" } + })), + summary, + findings + }; +} + +function normalize(f, dirByPath, diffMap) { + const file = f.file.replace(/\\/g, "/"); + const dir = [...dirByPath.keys()].find((p) => file === p || file.startsWith(p + "/")) ?? file.split("/").slice(0, 2).join("/"); + const line = Number(f.line) || 0; + const inDiff = diffMap ? (line > 0 ? diffMap.get(file)?.has(line) ?? false : null) : null; + const sourceLine = line > 0 ? readLine(file, line) : null; + return { + id: `${f.source}:${f.rule_id}:${file}:${f.json_path ?? line}`, + source: f.source, + rule_id: f.rule_id, + severity: f.severity, + effective_severity: f.severity, + downgrade_reason: null, + promote_reason: null, + fix_strategy: f.fix_strategy, + message: f.message, + why: f.why ?? null, + file, + dir, + line, + end_line: f.end_line ?? null, + json_path: f.json_path ?? null, + snippet: f.snippet ?? null, + target_text: f.target_text ?? null, + suggested_replacement: f.suggested_replacement ?? null, + replacement_context: f.replacement_context ?? null, + source_line: sourceLine, + in_diff: inDiff, + doc_url: docUrl(f.rule_id) + }; +} + +function applyPolicy(f, dirInfo, mode, diffMap) { + if (dirInfo?.section === "catalog" && f.severity === "ADVICE") { + f.effective_severity = "ACTION"; + f.promote_reason = "catalog-strict"; + } + if (dirInfo?.status === "modified" && diffMap && f.effective_severity === "ACTION" && f.in_diff === false && f.rule_id !== "ArcaneAuditorError") { + f.effective_severity = "ADVICE"; + f.promote_reason = null; + f.downgrade_reason = "pre-existing-line"; + } + if (mode === "advisory" && f.effective_severity === "ACTION") { + f.effective_severity = "ADVICE"; + f.promote_reason = null; + f.downgrade_reason = f.downgrade_reason ?? "advisory-mode"; + } +} + +// Whether a finding can become a one-click GitHub suggestion. +export function suggestionFor(f) { + if (f.fix_strategy !== "actionable" || f.suggested_replacement == null || f.source_line == null) return null; + if (f.replacement_context === "full_line") return f.suggested_replacement; + if (!f.target_text || !["substring", "full_field"].includes(f.replacement_context)) return null; + if (!f.source_line.includes(f.target_text)) return null; + if (/\r|\n/.test(f.suggested_replacement)) return null; + + let target = f.target_text; + let replacement = f.suggested_replacement; + // Arcane suggests "<% apiGatewayEndpoint + '/path' %>" for a hardcoded + // URL. When the URL already sits inside a script expression, nesting a + // second <% %> is wrong: swap the quoted literal for the inner expression. + const inner = replacement.match(/^<%\s*(.*?)\s*%>$/); + if (inner && f.source_line.includes("<%")) { + const quoted = [`'${target}'`, `"${target}"`].find((q) => f.source_line.includes(q)); + if (!quoted) return null; + target = quoted; + replacement = inner[1]; + } + // A bare expression (site.applicationId) offered for text that sits inside + // a quoted script literal must be spliced in as concatenation, otherwise + // the "fix" is still a string. + if (/^[A-Za-z_][\w.]*$/.test(replacement) && f.source_line.includes("<%")) { + const m = f.source_line.match(new RegExp(`(['"])([^'"]*)${escapeRe(target)}([^'"]*)\\1`)); + if (m) { + const [whole, q, pre, post] = m; + const parts = []; + if (pre) parts.push(`${q}${pre}${q}`); + parts.push(replacement); + if (post) parts.push(`${q}${post}${q}`); + return f.source_line.replace(whole, parts.join(" + ")); + } + } + return f.source_line.replace(target, replacement); +} + +function escapeRe(s) { + return String(s).replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); +} + +function readLine(file, line) { + const abs = join(repoRoot, file); + if (!existsSync(abs)) return null; + try { + const lines = readFileSync(abs, "utf8").split(/\r?\n/); + return lines[line - 1] ?? null; + } catch { + return null; + } +} + +function rank(f) { + return f.effective_severity === "ACTION" ? 0 : 1; +} +function cmp(a, b) { + return a < b ? -1 : a > b ? 1 : 0; +} +function clip(s, n) { + if (s == null) return null; + s = String(s); + return s.length > n ? s.slice(0, n) + "…" : s; +} diff --git a/scripts/install-arcane.sh b/scripts/install-arcane.sh new file mode 100755 index 0000000..01cf158 --- /dev/null +++ b/scripts/install-arcane.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# Installs the Arcane Auditor CLI into .arcane-auditor/bin/ (gitignored) so +# `node scripts/audit-examples.mjs` can run the same audit CI runs. +# +# Uses the install script that ships with Arcane's GitHub Action, pinned to +# the same commit the CI workflow uses, so local and CI installs match. +set -euo pipefail + +root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +pin="$root/.arcane-auditor/action-ref" +ref="$(tr -d '[:space:]' < "$pin")" + +url="https://raw.githubusercontent.com/${ref%@*}/${ref#*@}/.github/action/install.sh" +echo "Fetching installer from $url" +tmp="$(mktemp)" +trap 'rm -f "$tmp"' EXIT +curl -fsSL -o "$tmp" "$url" + +ARCANE_INSTALL_DIR="$root/.arcane-auditor/bin" bash "$tmp" +echo +echo "Now run: node scripts/audit-examples.mjs --changed" diff --git a/scripts/validate-examples.mjs b/scripts/validate-examples.mjs index 19152f5..6a2d7cb 100644 --- a/scripts/validate-examples.mjs +++ b/scripts/validate-examples.mjs @@ -9,127 +9,158 @@ import { existsSync, readFileSync, readdirSync, statSync, writeFileSync } from " import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; -const repoRoot = join(dirname(fileURLToPath(import.meta.url)), ".."); +export const repoRoot = join(dirname(fileURLToPath(import.meta.url)), ".."); const readmePath = join(repoRoot, "README.md"); -const checkOnly = process.argv.includes("--check"); -const config = JSON.parse(readFileSync(join(repoRoot, "hub.config.json"), "utf8")); +export const config = JSON.parse(readFileSync(join(repoRoot, "hub.config.json"), "utf8")); // catalog/ holds Workday-built apps, examples/ holds community examples. -const sections = [ +export const sections = [ { dir: "catalog", markers: "catalog", defaultSource: "workday" }, { dir: "examples", markers: "examples", defaultSource: "community" } ]; -const errors = []; -const entries = []; +// Validates one entry folder (catalog/ or examples/). Returns +// { errors, entry }: entry is null when the folder has no usable metadata. +// Also used by scripts/audit/hub-rules.mjs, so keep it free of side effects. +export function validateEntry(sectionDir, name) { + const section = sections.find((s) => s.dir === sectionDir); + const dir = join(repoRoot, sectionDir, name); + const errors = []; -for (const section of sections) { - const sectionDir = join(repoRoot, section.dir); - if (!existsSync(sectionDir)) continue; - - for (const name of readdirSync(sectionDir).sort()) { - // _template and dotfiles are not entries - if (name.startsWith("_") || name.startsWith(".")) continue; - const dir = join(sectionDir, name); - if (!statSync(dir).isDirectory()) continue; - - if (!existsSync(join(dir, "README.md"))) { - errors.push(`${section.dir}/${name}: missing README.md`); - } + if (!existsSync(join(dir, "README.md"))) { + errors.push(`${sectionDir}/${name}: missing README.md`); + } - const metaPath = join(dir, "example.json"); - if (!existsSync(metaPath)) { - errors.push(`${section.dir}/${name}: missing example.json`); - continue; - } + const metaPath = join(dir, "example.json"); + if (!existsSync(metaPath)) { + errors.push(`${sectionDir}/${name}: missing example.json`); + return { errors, entry: null }; + } - let meta; - try { - meta = JSON.parse(readFileSync(metaPath, "utf8")); - } catch (err) { - errors.push(`${section.dir}/${name}/example.json is not valid JSON: ${err.message}`); - continue; - } + let meta; + try { + meta = JSON.parse(readFileSync(metaPath, "utf8")); + } catch (err) { + errors.push(`${sectionDir}/${name}/example.json is not valid JSON: ${err.message}`); + return { errors, entry: null }; + } - if (!meta.title) errors.push(`${section.dir}/${name}: example.json needs a "title"`); - if (!meta.description) errors.push(`${section.dir}/${name}: example.json needs a "description"`); + if (!meta.title) errors.push(`${sectionDir}/${name}: example.json needs a "title"`); + if (!meta.description) errors.push(`${sectionDir}/${name}: example.json needs a "description"`); - if (!meta.type) { - errors.push(`${section.dir}/${name}: example.json needs a "type"`); - } else if (!config.types.includes(meta.type)) { - errors.push(`${section.dir}/${name}: "${meta.type}" is not an approved type. Pick from: ${config.types.join(", ")}`); - } + if (!meta.type) { + errors.push(`${sectionDir}/${name}: example.json needs a "type"`); + } else if (!config.types.includes(meta.type)) { + errors.push(`${sectionDir}/${name}: "${meta.type}" is not an approved type. Pick from: ${config.types.join(", ")}`); + } - for (const component of asList(meta.components)) { - if (!config.components.includes(component)) { - errors.push(`${section.dir}/${name}: "${component}" is not an approved component. Pick from: ${config.components.join(", ")}`); - } + for (const component of asList(meta.components)) { + if (!config.components.includes(component)) { + errors.push(`${sectionDir}/${name}: "${component}" is not an approved component. Pick from: ${config.components.join(", ")}`); } + } - for (const product of asList(meta.products)) { - if (!config.products.includes(product)) { - errors.push(`${section.dir}/${name}: "${product}" is not an approved product. Pick from: ${config.products.join(", ")}`); - } + for (const product of asList(meta.products)) { + if (!config.products.includes(product)) { + errors.push(`${sectionDir}/${name}: "${product}" is not an approved product. Pick from: ${config.products.join(", ")}`); } + } - if (meta.tutorial && !meta.tutorial.startsWith("https://")) { - errors.push(`${section.dir}/${name}: "tutorial" should be an https link, or left out`); - } + if (meta.tutorial && !meta.tutorial.startsWith("https://")) { + errors.push(`${sectionDir}/${name}: "tutorial" should be an https link, or left out`); + } - if (meta.source && !["workday", "community"].includes(meta.source)) { - errors.push(`${section.dir}/${name}: "source" must be "workday" or "community", or left out`); - } - if (section.dir === "catalog" && meta.source === "community") { - errors.push(`catalog/${name}: catalog apps are Workday-maintained, so "source" cannot be "community". Community submissions live in examples/.`); - } + if (meta.source && !["workday", "community"].includes(meta.source)) { + errors.push(`${sectionDir}/${name}: "source" must be "workday" or "community", or left out`); + } + if (sectionDir === "catalog" && meta.source === "community") { + errors.push(`catalog/${name}: catalog apps are Workday-maintained, so "source" cannot be "community". Community submissions live in examples/.`); + } - entries.push({ + return { + errors, + entry: { id: name, sectionMarkers: section.markers, - path: `${section.dir}/${name}`, + path: `${sectionDir}/${name}`, title: meta.title || name, description: meta.description || "", type: meta.type || "" - }); - } + } + }; } -if (errors.length > 0) { - console.error("Problems found:\n"); - for (const error of errors) console.error(` - ${error}`); - console.error(`\n${errors.length} problem(s). Fix them and re-run.`); - process.exit(1); -} +// Validates every entry in both sections. Returns { errors, entries }. +export function validateAll() { + const errors = []; + const entries = []; + for (const section of sections) { + const sectionDir = join(repoRoot, section.dir); + if (!existsSync(sectionDir)) continue; -entries.sort((a, b) => a.title.localeCompare(b.title)); + for (const name of readdirSync(sectionDir).sort()) { + // _template and dotfiles are not entries + if (name.startsWith("_") || name.startsWith(".")) continue; + if (!statSync(join(sectionDir, name)).isDirectory()) continue; -const readme = readFileSync(readmePath, "utf8"); + const result = validateEntry(section.dir, name); + errors.push(...result.errors); + if (result.entry) entries.push(result.entry); + } + } + return { errors, entries }; +} -// Compare each table by content, not formatting, so tools like Prettier -// can reflow them without the check calling them stale. -let allInSync = true; -for (const section of sections) { - const expected = rowsFor(section.markers); - const current = tableRows(readme, section.markers); - if (JSON.stringify(current) !== JSON.stringify(expected)) allInSync = false; +// Compares the README index tables against the entries. Returns true when +// every table is in sync. +export function tablesInSync(entries, readme = readFileSync(readmePath, "utf8")) { + for (const section of sections) { + const expected = rowsFor(entries, section.markers); + const current = tableRows(readme, section.markers); + if (JSON.stringify(current) !== JSON.stringify(expected)) return false; + } + return true; } -if (checkOnly) { - if (!allInSync) { - console.error("A README table is out of date. Run: node scripts/validate-examples.mjs"); +const isMain = process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]; +if (isMain) main(); + +function main() { + const checkOnly = process.argv.includes("--check"); + const { errors, entries } = validateAll(); + + if (errors.length > 0) { + console.error("Problems found:\n"); + for (const error of errors) console.error(` - ${error}`); + console.error(`\n${errors.length} problem(s). Fix them and re-run.`); process.exit(1); } - console.log(`OK: ${entries.length} entr${entries.length === 1 ? "y" : "ies"} validated, README tables in sync.`); -} else if (allInSync) { - console.log(`Validated ${entries.length} entr${entries.length === 1 ? "y" : "ies"}, README tables already up to date.`); -} else { - let updated = readme; - for (const section of sections) { - updated = withFreshTable(updated, section.markers); + + entries.sort((a, b) => a.title.localeCompare(b.title)); + + const readme = readFileSync(readmePath, "utf8"); + + // Compare each table by content, not formatting, so tools like Prettier + // can reflow them without the check calling them stale. + const allInSync = tablesInSync(entries, readme); + + if (checkOnly) { + if (!allInSync) { + console.error("A README table is out of date. Run: node scripts/validate-examples.mjs"); + process.exit(1); + } + console.log(`OK: ${entries.length} entr${entries.length === 1 ? "y" : "ies"} validated, README tables in sync.`); + } else if (allInSync) { + console.log(`Validated ${entries.length} entr${entries.length === 1 ? "y" : "ies"}, README tables already up to date.`); + } else { + let updated = readme; + for (const section of sections) { + updated = withFreshTable(updated, entries, section.markers); + } + writeFileSync(readmePath, updated); + console.log(`Validated ${entries.length} entr${entries.length === 1 ? "y" : "ies"} and updated README.md.`); } - writeFileSync(readmePath, updated); - console.log(`Validated ${entries.length} entr${entries.length === 1 ? "y" : "ies"} and updated README.md.`); } function asList(value) { @@ -138,7 +169,7 @@ function asList(value) { return []; } -function rowsFor(markers) { +function rowsFor(entries, markers) { return entries .filter((entry) => entry.sectionMarkers === markers) .map((entry) => [`[\`${entry.id}\`](${entry.path})`, entry.description, entry.type]); @@ -170,9 +201,9 @@ function tableRows(text, markers) { return rows; } -function withFreshTable(text, markers) { +function withFreshTable(text, entries, markers) { const [startAt, endAt] = markerPositions(text, markers); - const rows = rowsFor(markers).map((cells) => `| ${cells.join(" | ")} |`); + const rows = rowsFor(entries, markers).map((cells) => `| ${cells.join(" | ")} |`); const table = ["| Example | Description | Type |", "| --- | --- | --- |", ...rows].join("\n"); return text.slice(0, startAt) + "\n" + table + "\n" + text.slice(endAt); }