Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 20 additions & 1 deletion DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1081,7 +1081,7 @@ vg scan [path] [--vulns] [--full] [--format text|json|sarif|md] [--out <file>] [
| `--format` | `text` | Output format: `text`, `json`, `sarif`, or `md` |
| `--out <file>` | — | Write output to a file |
| `--fail-on <level>` | — | Exit with code 2 if findings at this level exist. `warn` / `error` gate on drift findings. `architecture-finding` (hard boundary violations) and `architecture-warning` (violations and warnings) gate on the architecture module's boundary findings, judged under the policy pack in force — `hexagonal-v1` unless `.vibgrate/architecture.toml`, `VIBGRATE_ARCHITECTURE_POLICY` or `vg build --policy` says `layered-v1`. The output names the pack whether the gate passes or fails; each failing row is `file:line symbol violation: … (rule)`. Pick the pack before turning this on: see [Architecture policy packs](./docs/architecture-policies.md) |
| `--baseline <file>` | — | Compare against a previous baseline |
| `--baseline <file>` | — | Compare against a previous baseline. Matched findings stay in the report and are listed on `baseline.suppressed` (and as SARIF suppressions); the text and Markdown reports include the count |
| `--changed-only` | — | Only scan changed files |
| `--concurrency <n>` | `8` | Max concurrent npm registry calls |
| `--drift-budget <score>` | — | Fitness gate: fail if drift score is above this budget |
Expand Down Expand Up @@ -2808,6 +2808,21 @@ Recommended workflow:

This makes drift a formal quality gate (fitness function), not just reporting.

`vg scan --baseline` does not drop findings that were already in the baseline. The scan artifact records them so a suppression is visible in the same document:

```json
"baseline": {
"compared": true,
"file": ".vibgrate/baseline.json",
"suppressedCount": 2,
"suppressed": [
{ "ruleId": "vibgrate/dependency-rot", "location": "package.json", "id": "af173ea1c1efa4fd" }
]
}
```

`suppressed` is sorted by `ruleId`, then `location`, then `id`. `file` is the repo-relative path (`/` separators), or the file's basename when the baseline sits outside the repo — an absolute path is never stored. `id` is 16 hex characters of SHA-256 over the rule, the location, and a stable subject: advisory id, ecosystem, and package for a vulnerability; the package or framework name for a major-lag finding; otherwise the rule and location. Text and Markdown reports include `suppressedCount`. SARIF keeps the same results and sets `suppressions[].properties.id` to that id.

## DriftScore

### How the Score Is Calculated
Expand Down Expand Up @@ -2854,10 +2869,14 @@ The default output. A coloured, human-readable report showing:

The full scan artifact in JSON format. Contains all raw data, scores, findings, and VCS metadata. Stable schema (`schemaVersion: "1.0"`). This is the same artifact saved to `.vibgrate/scan_result.json`.

When the scan was run with `--baseline`, the artifact also contains a `baseline` object: `compared`, `file`, `suppressedCount`, and `suppressed` (`ruleId`, `location`, `id`). Those entries are the findings that were already in the baseline. The same findings remain in `findings`. See [Drift Baselines & Fitness Functions](#drift-baselines--fitness-functions) for the field meanings and the id algorithm.

### SARIF

[Static Analysis Results Interchange Format](https://sarifweb.azurewebsites.net/) — compatible with GitHub Code Scanning and Azure DevOps. Contains findings only (not all metrics). Ideal for integrating drift findings directly into your PR review workflow.

A `--baseline` scan keeps every result. Each result that matches the baseline gets a SARIF `suppressions` entry whose `properties.id` is the same id as `baseline.suppressed` in the JSON artifact. The run invocation records `baselineCompared` and `baselineSuppressedCount`, including when the count is zero.

### Markdown

A clean Markdown report suitable for PRs, wikis, or documentation.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -557,6 +557,7 @@ vg scan --baseline .vibgrate/baseline.json --drift-budget 40 --drift-worsening 5

- `--drift-budget <score>` fails the build if drift exceeds your budget.
- `--drift-worsening <percent>` fails the build if drift worsens by more than X% vs baseline.
- `--baseline <file>` keeps every finding and records which ones were already in the baseline (`baseline.suppressed` in JSON, the same ids on SARIF suppressions). The text report includes the count.

Copy-paste CI templates live in `examples/github-actions/`. Azure DevOps and GitLab CI snippets are in [DOCS.md](./DOCS.md#ci-integration).

Expand Down
181 changes: 181 additions & 0 deletions src/core-open/baseline-audit.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
// VENDORED from @vibgrate/core-open (packages/vibgrate-core-open) by
// scripts/vendor-core-open.mjs. Do not edit here — change the source package
// and re-run the vendor script. Apache-2.0.
import { createHash } from 'node:crypto';
import * as path from 'node:path';
import type { BaselineComparison, BaselineSuppressedFinding, Finding } from './types.js';

/**
* Rules that emit at most one finding per location. Their id ignores the
* message, so a version-number edit does not look like a new finding.
*/
const LOCATION_ONLY_RULES = new Set([
'vibgrate/runtime-eol',
'vibgrate/runtime-lag',
'vibgrate/dependency-rot',
]);

const MAJOR_LAG_RULES = new Set([
'vibgrate/framework-major-lag',
'vibgrate/dependency-major-lag',
]);

/** "React is 2 major versions behind …" / "lodash is 3 major versions behind …" */
const MAJOR_LAG_NAME = /^(.+?) is \d+ major versions behind/;

/**
* Content-derived id for a drift finding.
*
* First 16 hex characters of SHA-256 over `[ruleId, location, subject]`.
* The subject keeps two findings at the same path distinct without copying
* the message (which changes as versions move, and which must not be repeated
* into the suppression record):
*
* - vulnerability findings: `advisoryId`, `ecosystem`, and `package` from `details`
* - framework / dependency major-lag: the name that precedes "is N major versions behind"
* - runtime EOL, runtime lag, dependency rot: nothing (rule + location is enough)
* - any other rule: the message, so distinct findings do not collapse
*/
export function findingId(finding: Finding): string {
const payload = JSON.stringify([finding.ruleId, finding.location, findingSubject(finding)]);
return createHash('sha256').update(payload).digest('hex').slice(0, 16);
}

/**
* Current findings whose id also appears in the baseline.
* Sorted by ruleId, then location, then id. The input arrays are not modified,
* and the result never includes the finding message.
*/
export function suppressedBaselineFindings(
current: readonly Finding[],
baseline: readonly Finding[],
): BaselineSuppressedFinding[] {
const baselineIds = new Set<string>();
for (const finding of baseline) {
const normalized = asFinding(finding);
if (normalized) baselineIds.add(findingId(normalized));
}

const suppressed: BaselineSuppressedFinding[] = [];
const seen = new Set<string>();
for (const finding of current) {
const normalized = asFinding(finding);
if (!normalized) continue;
const id = findingId(normalized);
if (!baselineIds.has(id) || seen.has(id)) continue;
seen.add(id);
suppressed.push({ ruleId: normalized.ruleId, location: normalized.location, id });
}

suppressed.sort(compareSuppressed);
return suppressed;
}

/**
* Reference to the baseline file that is safe to store in a scan artifact.
* Repo-relative, with `/` separators. A file outside the repo contributes
* only its basename, so a home-directory path never enters the document.
*/
export function baselineFileReference(rootDir: string, baselinePath: string): string {
const absRoot = path.resolve(rootDir);
const absBaseline = path.resolve(baselinePath);
const rel = path.relative(absRoot, absBaseline);
if (!rel || rel.startsWith('..') || path.isAbsolute(rel)) {
return path.basename(absBaseline);
}
return rel.split(path.sep).join('/');
}

/** "1 finding suppressed" / "2 findings suppressed". */
export function baselineSuppressionPhrase(count: number): string {
const n = Number.isFinite(count) ? count : 0;
return `${n} ${n === 1 ? 'finding' : 'findings'} suppressed`;
}

/**
* Read a `baseline` block from a scan artifact. Returns undefined for the
* legacy string path and for anything that is not a comparison record, so
* older artifacts still format.
*/
export function readBaselineComparison(value: unknown): BaselineComparison | undefined {
if (!value || typeof value !== 'object') return undefined;
const raw = value as Record<string, unknown>;
if (raw.compared !== true || !Array.isArray(raw.suppressed)) return undefined;

const suppressed: BaselineSuppressedFinding[] = [];
for (const item of raw.suppressed) {
if (!item || typeof item !== 'object') continue;
const row = item as Record<string, unknown>;
if (typeof row.ruleId !== 'string' || typeof row.location !== 'string' || typeof row.id !== 'string') continue;
suppressed.push({ ruleId: row.ruleId, location: row.location, id: row.id });
}

const suppressedCount = typeof raw.suppressedCount === 'number' && Number.isFinite(raw.suppressedCount)
? raw.suppressedCount
: suppressed.length;

return {
compared: true,
file: typeof raw.file === 'string' ? raw.file : '',
suppressedCount,
suppressed,
};
}

/** Phrase for text/markdown, or undefined when this artifact did not compare a baseline. */
export function baselineSuppressionPhraseFrom(baseline: unknown): string | undefined {
const comparison = readBaselineComparison(baseline);
if (!comparison) return undefined;
return baselineSuppressionPhrase(comparison.suppressedCount);
}

function findingSubject(finding: Finding): string {
const fromDetails = subjectFromDetails(finding);
if (fromDetails) return fromDetails;

if (MAJOR_LAG_RULES.has(finding.ruleId)) {
const name = nameBeforeMajorLag(finding.message);
return name ? `name=${name}` : `message=${finding.message}`;
}

if (LOCATION_ONLY_RULES.has(finding.ruleId)) return '';
return finding.message ? `message=${finding.message}` : '';
}

function subjectFromDetails(finding: Finding): string {
const details = finding.details;
if (!details || typeof details.advisoryId !== 'string' || !details.advisoryId) return '';
const ecosystem = typeof details.ecosystem === 'string' ? details.ecosystem : '';
const pkg = typeof details.package === 'string' ? details.package : '';
return `advisoryId=${details.advisoryId}\necosystem=${ecosystem}\npackage=${pkg}`;
}

function nameBeforeMajorLag(message: string): string {
const match = MAJOR_LAG_NAME.exec(message);
return match?.[1]?.trim() ?? '';
}

function asFinding(value: unknown): Finding | undefined {
if (!value || typeof value !== 'object') return undefined;
const raw = value as Partial<Finding>;
if (typeof raw.ruleId !== 'string' || typeof raw.location !== 'string') return undefined;
const level = raw.level === 'error' || raw.level === 'note' || raw.level === 'warning' ? raw.level : 'warning';
const finding: Finding = {
ruleId: raw.ruleId,
level,
message: typeof raw.message === 'string' ? raw.message : '',
location: raw.location,
};
if (raw.details && typeof raw.details === 'object') finding.details = raw.details;
return finding;
}

function compareSuppressed(a: BaselineSuppressedFinding, b: BaselineSuppressedFinding): number {
return cmp(a.ruleId, b.ruleId) || cmp(a.location, b.location) || cmp(a.id, b.id);
}

function cmp(a: string, b: string): number {
if (a < b) return -1;
if (a > b) return 1;
return 0;
}
7 changes: 7 additions & 0 deletions src/core-open/formatters/markdown.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// scripts/vendor-core-open.mjs. Do not edit here — change the source package
// and re-run the vendor script. Apache-2.0.
import type { ScanArtifact } from '../types.js';
import { baselineSuppressionPhraseFrom } from '../baseline-audit.js';
import { securityPacksLabel } from './text.js';

/** Rows shown before the infrastructure-findings table is cut with an "… N more" line. */
Expand Down Expand Up @@ -186,5 +187,11 @@ export function formatMarkdown(artifact: ScanArtifact): string {
lines.push('');
}

const baselinePhrase = baselineSuppressionPhraseFrom(artifact.baseline);
if (baselinePhrase) {
lines.push(`**Baseline:** ${baselinePhrase}`);
lines.push('');
}

return lines.join('\n');
}
48 changes: 40 additions & 8 deletions src/core-open/formatters/sarif.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,25 @@
// scripts/vendor-core-open.mjs. Do not edit here — change the source package
// and re-run the vendor script. Apache-2.0.
import type { ScanArtifact, Finding, SecurityFinding, SecuritySection, SecuritySeverity } from '../types.js';
import { findingId, readBaselineComparison } from '../baseline-audit.js';

/**
* Generate a SARIF 2.1.0 document from scan artifact.
*
* The first run carries the drift findings and is byte-for-byte what it has
* always been. When the artifact carries security-pack findings
* (`extended.security`, from `vg scan --iac`), a second run is appended for
* them — one run per artifact, with every pack listed under
* `tool.extensions`, so a scan that ran no pack produces exactly the same
* bytes as before.
* The first run carries the drift findings. With no baseline comparison and
* no security-pack findings, that run is byte-for-byte what it has always
* been. A baseline comparison keeps every result and adds `suppressions`
* whose `properties.id` matches `baseline.suppressed` in the JSON artifact,
* plus an invocation property for the count (including zero). When the
* artifact carries security-pack findings (`extended.security`, from
* `vg scan --iac`), a second run is appended for them — one run per artifact,
* with every pack listed under `tool.extensions`.
*/
export function formatSarif(artifact: ScanArtifact): object {
const comparison = readBaselineComparison(artifact.baseline);
const suppressedIds = comparison ? new Set(comparison.suppressed.map((row) => row.id)) : null;
const rules = buildRules(artifact.findings);
const results = artifact.findings.map((f) => toSarifResult(f));
const results = artifact.findings.map((f) => toSarifResult(f, suppressedIds));

const runs: object[] = [
{
Expand All @@ -32,6 +37,17 @@ export function formatSarif(artifact: ScanArtifact): object {
{
executionSuccessful: true,
startTimeUtc: artifact.timestamp,
// Present only after a baseline comparison, so a scan that did not
// compare stays byte-for-byte what it was. A count of zero is still
// recorded: comparison with nothing matched is not a silent omission.
...(comparison
? {
properties: {
baselineCompared: true,
baselineSuppressedCount: comparison.suppressedCount,
},
}
: {}),
},
],
},
Expand Down Expand Up @@ -179,7 +195,9 @@ function buildRules(findings: Finding[]) {
});
}

function toSarifResult(finding: Finding) {
function toSarifResult(finding: Finding, suppressedIds: ReadonlySet<string> | null) {
const id = suppressedIds ? findingId(finding) : undefined;
const suppressed = id !== undefined && suppressedIds?.has(id) === true;
return {
ruleId: finding.ruleId,
level: finding.level === 'error' ? 'error' : finding.level === 'warning' ? 'warning' : 'note',
Expand All @@ -196,5 +214,19 @@ function toSarifResult(finding: Finding) {
// Surface structured finding detail (e.g. advisory id, CVSS, fixed version)
// to consumers like GitHub code scanning without bloating the message text.
...(finding.details && Object.keys(finding.details).length > 0 ? { properties: finding.details } : {}),
// The result stays in `results`. The suppression carries the same id as
// `baseline.suppressed` in the JSON artifact.
...(suppressed && id
? {
suppressions: [
{
kind: 'external',
status: 'accepted',
justification: 'Matched a finding recorded in the drift baseline.',
properties: { id },
},
],
}
: {}),
};
}
25 changes: 16 additions & 9 deletions src/core-open/formatters/text.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
// and re-run the vendor script. Apache-2.0.
import chalk from 'chalk';
import type { ScanArtifact, BillingSummary, ExtendedScanResults, InventoryItem, ServiceDependencyItem, ArchitectureResult, SecurityFinding, SecuritySection } from '../types.js';
import { baselineSuppressionPhraseFrom } from '../baseline-audit.js';
import { driftBar } from '../ui/bar.js';
import { titleBox, panelBox } from '../ui/box.js';

Expand Down Expand Up @@ -85,15 +86,21 @@ export function formatText(artifact: ScanArtifact, opts: FormatTextOptions = {})
lines.push('');
}

if (artifact.delta !== undefined) {
// Drift is "lower is better": a negative delta means drift fell (good),
// a positive delta means drift rose (bad).
const deltaStr = artifact.delta < 0
? chalk.green(`${artifact.delta}`)
: artifact.delta > 0
? chalk.red(`+${artifact.delta}`)
: chalk.dim('0');
lines.push(chalk.bold(' Drift Delta: ') + deltaStr + ' (vs baseline)');
const baselinePhrase = baselineSuppressionPhraseFrom(artifact.baseline);
if (artifact.delta !== undefined || baselinePhrase) {
if (artifact.delta !== undefined) {
// Drift is "lower is better": a negative delta means drift fell (good),
// a positive delta means drift rose (bad).
const deltaStr = artifact.delta < 0
? chalk.green(`${artifact.delta}`)
: artifact.delta > 0
? chalk.red(`+${artifact.delta}`)
: chalk.dim('0');
lines.push(chalk.bold(' Drift Delta: ') + deltaStr + ' (vs baseline)');
}
if (baselinePhrase) {
lines.push(chalk.bold(' Baseline: ') + baselinePhrase);
}
lines.push('');
}

Expand Down
10 changes: 10 additions & 0 deletions src/core-open/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,16 @@ export { formatText } from './formatters/text.js';
export { formatSarif } from './formatters/sarif.js';
export { formatMarkdown } from './formatters/markdown.js';

// ── Baseline comparison (audit record; findings are not dropped) ────────────
export {
baselineFileReference,
baselineSuppressionPhrase,
baselineSuppressionPhraseFrom,
findingId,
readBaselineComparison,
suppressedBaselineFindings,
} from './baseline-audit.js';

// ── Scanners (fact collection) ───────────────────────────────────────────────
export { scanNodeProjects } from './scanners/node-scanner.js';
export { scanDotnetProjects } from './scanners/dotnet-scanner.js';
Expand Down
Loading
Loading