Skip to content

Commit 816fbbe

Browse files
authored
Merge pull request #8 from Workday/auditor-integration
Audit example PRs with Arcane Auditor and hub rules
2 parents 487d736 + 0257f94 commit 816fbbe

21 files changed

Lines changed: 2429 additions & 94 deletions

.arcane-auditor/README.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Arcane Auditor configuration
2+
3+
This folder configures the example audit that runs on every pull request
4+
(`.github/workflows/audit-examples.yml`) and locally through
5+
`node scripts/audit-examples.mjs`.
6+
7+
| File | Purpose |
8+
| --- | --- |
9+
| `config.json` | Rule configuration for [Arcane Auditor](https://github.com/Developers-and-Dragons/ArcaneAuditor). Generated with `ArcaneAuditorCLI generate-config`, then adjusted (see below). |
10+
| `action-ref` | The Arcane GitHub Action revision the CI workflow uses, as `owner/repo@ref`. `scripts/install-arcane.sh` reads it so local installs match CI. |
11+
| `bin/` | Local CLI install, gitignored. Created by `scripts/install-arcane.sh`. |
12+
13+
The hub's own rules (folder naming, `example.json`, README sections, period
14+
literals, app reference ids) live in `scripts/audit/hub-rules.mjs`, not here.
15+
16+
## Rule policy
17+
18+
Arcane ships 48 rules in two tiers. ACTION findings fail the audit check,
19+
ADVICE findings are shown as suggestions. `config.json` keeps every rule
20+
enabled and changes three things for a public examples hub:
21+
22+
| Rule | Change | Why |
23+
| --- | --- | --- |
24+
| `HardcodedApplicationIdRule` | ADVICE to ACTION | Examples exist to be copied. A hardcoded app id guarantees the copy breaks, and the fix is mechanical (`site.applicationId`). |
25+
| `OrchestrationGlobalErrorHandlerRule` | ACTION to ADVICE | Error-handler scaffolding is not always the lesson an orchestration example teaches. Worth suggesting, not worth blocking. |
26+
| `OrchestrationApiStepErrorHandlerRule` | ACTION to ADVICE | Same reasoning. |
27+
| `PMDSectionOrderingRule` | fix strategy to `human_review` | Arcane v2.0.0 has no automatic fix payload for this rule; marking it actionable produced empty suggestions. |
28+
29+
Everything else runs at Arcane's default severity. Prefer downgrading a rule
30+
over disabling it, so the best-practices doc can still explain it.
31+
32+
Policy applied on top by `scripts/audit/report.mjs`:
33+
34+
- `catalog/` folders are held to the stricter bar: ADVICE counts as ACTION.
35+
- In a folder that already existed, ACTION findings on lines the PR did not
36+
touch are downgraded so contributors are only blocked on what they wrote.
37+
- The `audit-override` label, or the repository variable `AUDIT_MODE=advisory`,
38+
turns the whole check advisory.
39+
40+
## Bumping Arcane
41+
42+
1. Pick the release on the Arcane releases page and note its Linux and macOS
43+
CLI asset hashes (`sha256sum` the downloads yourself).
44+
2. In the Arcane action repository, add the hashes to
45+
`.github/action/install.sh` and bump the default `version` in `action.yml`.
46+
Push, note the commit sha.
47+
3. Update `action-ref` here and the `uses:` line in
48+
`.github/workflows/audit-examples.yml` to that sha.
49+
4. Regenerate `config.json` if the rule set changed:
50+
`ArcaneAuditorCLI generate-config -o /tmp/new.json`, diff against the
51+
current file, and re-apply the overrides above.
52+
5. Re-run the regression check below.
53+
54+
## Regression check
55+
56+
Pull request [#7](https://github.com/Workday/WorkdayDeveloperProgram/pull/7)
57+
(`examples/Promotion_Nomination`) is the reference case: it contains most of
58+
the mistakes the audit exists to catch. To re-run it:
59+
60+
```bash
61+
git fetch origin pull/7/head:pr-7
62+
git worktree add /tmp/pr-7 pr-7
63+
cp -R scripts .arcane-auditor/config.json /tmp/pr-7/ # bring the current audit scripts along
64+
cd /tmp/pr-7 && node scripts/audit-examples.mjs --changed main HEAD
65+
```
66+
67+
Expected with Arcane v2.0.0 and the current hub rules (62 findings, 32
68+
blocking):
69+
70+
Hub rules
71+
72+
- `HubFolderKebabCaseRule`: `Promotion_Nomination`
73+
- `HubExampleJsonRule`: `example.json` missing (hint: rename `app-info.json`), and `app-info.json` line 1 is the literal word `JSON`
74+
- `HubReadmeSectionsRule`: README is raw HTML, none of the four sections
75+
- `HubGitkeepRule`: `model/.gitkeep`, `presentation/.gitkeep`
76+
- `HubHardcodedPeriodLiteralRule`: `"2026-Q1"` in `managerNomination.pmd`
77+
- `HubAppReferenceIdRule`: `promotionNomination_rvylxm` in the `.amd` and `.smd`
78+
79+
Arcane, ACTION
80+
81+
- `HardcodedWorkdayAPIRule`: 3 endpoints across the two PMDs, 5 data providers in the `.amd`
82+
- `HardcodedApplicationIdRule`: the `.amd` data provider and the `submitPromotion` URL
83+
- `EndpointFailOnStatusCodesRule`: 8 endpoints
84+
- `WidgetIdRequiredRule`: 7 widgets
85+
- `ScriptConsoleLogRule`: 2 live `console.info` calls (the commented-out ones do not fire)
86+
87+
Arcane, ADVICE
88+
89+
- `ScriptVarUsageRule` (6), `ScriptStringConcatRule` (8), `EndpointBaseUrlTypeRule` (4), `StringBooleanRule` (2), `PMDSectionOrderingRule` (2), `ScriptComplexityRule` (1), `EndpointNameLowerCamelCaseRule` (1)
90+
- `ArcaneAuditorWarning`: Arcane's script parser gives up on one block in `managerNomination.pmd` (`var responseEmpData =getEmployeeData.invoke(`), so script rules are skipped for that block. That is an upstream grammar gap worth reporting.
91+
92+
Of the actionable findings, the hardcoded URL, console, `var`, and string
93+
boolean ones render as one-click suggestions on the PR. `failOnStatusCodes`
94+
insertions and multi-line string concatenations render as code blocks.

.arcane-auditor/action-ref

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Ekwuno/ArcaneAuditor@c31316d1147cb0e2d3f47688e668f7f3b2f9e887

0 commit comments

Comments
 (0)