Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 36 additions & 1 deletion DOCS.md
Original file line number Diff line number Diff line change
Expand Up @@ -1081,7 +1081,7 @@ vg scan [path] [--vulns] [--full] [--format text|json|sarif|md] [--out <file>] [
| `--format` | `text` | Output format: `text`, `json`, `sarif`, or `md` |
| `--out <file>` | — | Write output to a file |
| `--fail-on <level>` | — | Exit with code 2 if findings at this level exist. `warn` / `error` gate on drift findings. `architecture-finding` (hard boundary violations) and `architecture-warning` (violations and warnings) gate on the architecture module's boundary findings, judged under the policy pack in force — `hexagonal-v1` unless `.vibgrate/architecture.toml`, `VIBGRATE_ARCHITECTURE_POLICY` or `vg build --policy` says `layered-v1`. The output names the pack whether the gate passes or fails; each failing row is `file:line symbol violation: … (rule)`. Pick the pack before turning this on: see [Architecture policy packs](./docs/architecture-policies.md) |
| `--baseline <file>` | — | Compare against a previous baseline |
| `--baseline <file>` | — | Compare against a previous baseline. Matched findings stay in the report and are listed in `baselineComparison` (see [Drift Baselines](#drift-baselines--fitness-functions)) |
| `--changed-only` | — | Only scan changed files |
| `--concurrency <n>` | `8` | Max concurrent npm registry calls |
| `--drift-budget <score>` | — | Fitness gate: fail if drift score is above this budget |
Expand Down Expand Up @@ -2808,6 +2808,37 @@ Recommended workflow:

This makes drift a formal quality gate (fitness function), not just reporting.

### What a baseline comparison records

`vg scan --baseline` still reports the numeric drift delta (`delta`, and `--drift-worsening` uses that delta). It also writes an additive `baselineComparison` block on the scan artifact so a matched finding is not a silent drop. Findings stay in `findings`. The block is omitted when no baseline file was read.

```json
"baseline": ".vibgrate/baseline.json",
"delta": 2,
"baselineComparison": {
"compared": true,
"suppressedCount": 2,
"suppressed": [
{
"ruleId": "vibgrate/dependency-rot",
"location": "package.json",
"id": "c0ffee…"
}
]
}
```

| Field | Meaning |
| ----- | ------- |
| `baseline` | Repo-relative path of the file that was compared (basename if the file is outside the repo) |
| `baselineComparison.compared` | `true` when that file was read |
| `baselineComparison.suppressedCount` | How many current findings were already in the baseline |
| `baselineComparison.suppressed` | `{ ruleId, location, id }` for each match, sorted by `ruleId`, then `location`, then `id` |

`id` is 32 lowercase hex characters: the first 128 bits of SHA-256 over the length-prefixed `ruleId`, `level`, `location`, and `message`. The same finding always produces the same id. A changed message is a different finding and is not listed as suppressed. Text and Markdown reports include `Baseline suppressions: N` and mark matched rows `(baselined)`.

`baseline` remains the path string it has always been. `baselineComparison` is the new audit record.

## DriftScore

### How the Score Is Calculated
Expand Down Expand Up @@ -2854,10 +2885,14 @@ The default output. A coloured, human-readable report showing:

The full scan artifact in JSON format. Contains all raw data, scores, findings, and VCS metadata. Stable schema (`schemaVersion: "1.0"`). This is the same artifact saved to `.vibgrate/scan_result.json`.

When the scan compared a baseline, the artifact also carries `baseline` (the file path) and `baselineComparison` (`compared`, `suppressedCount`, and `suppressed` — see [What a baseline comparison records](#what-a-baseline-comparison-records)). Matched findings remain in `findings`.

### SARIF

[Static Analysis Results Interchange Format](https://sarifweb.azurewebsites.net/) — compatible with GitHub Code Scanning and Azure DevOps. Contains findings only (not all metrics). Ideal for integrating drift findings directly into your PR review workflow.

A finding that matched the baseline stays in `runs[0].results`. Its result gains a SARIF `suppressions` entry (`kind: "external"`, `status: "accepted"`) whose `properties.id` is the same id as `baselineComparison.suppressed`. Results that did not match have no `suppressions` field.

### Markdown

A clean Markdown report suitable for PRs, wikis, or documentation.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -557,6 +557,7 @@ vg scan --baseline .vibgrate/baseline.json --drift-budget 40 --drift-worsening 5

- `--drift-budget <score>` fails the build if drift exceeds your budget.
- `--drift-worsening <percent>` fails the build if drift worsens by more than X% vs baseline.
- Matched findings stay in the report. JSON records them in `baselineComparison` (`compared`, `suppressedCount`, `suppressed` sorted by `ruleId`, `location`, `id`), and SARIF puts those ids on `suppressions`. See [DOCS.md](./DOCS.md#what-a-baseline-comparison-records).

Copy-paste CI templates live in `examples/github-actions/`. Azure DevOps and GitLab CI snippets are in [DOCS.md](./DOCS.md#ci-integration).

Expand Down
93 changes: 93 additions & 0 deletions src/core-open/baseline-comparison.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import { createHash } from 'node:crypto';
import type { BaselineComparison, BaselineSuppressedFinding } from './types.js';

/**
* Fields that identify a drift finding across a baseline and a later scan.
* The message is part of the id, so a changed finding is not recorded as
* suppressed.
*/
export interface DriftFindingIdentity {
ruleId: string;
level: string;
location: string;
message: string;
}

/**
* Content id for a drift finding. 32 lowercase hex characters (128 bits of
* SHA-256). Parts are length-prefixed so concatenation cannot collide, and
* the digest does not depend on object key order, clock, or map iteration.
*/
export function driftFindingId(finding: DriftFindingIdentity): string {
const payload = [finding.ruleId, finding.level, finding.location, finding.message]
.map(encodePart)
.join('\n');
return createHash('sha256').update(payload).digest('hex').slice(0, 32);
}

function encodePart(value: string): string {
return `${Buffer.byteLength(value, 'utf8')}:${value}`;
}

function isIdentity(value: unknown): value is DriftFindingIdentity {
if (!value || typeof value !== 'object') return false;
const finding = value as Record<string, unknown>;
return typeof finding.ruleId === 'string'
&& typeof finding.level === 'string'
&& typeof finding.location === 'string'
&& typeof finding.message === 'string';
}

/**
* Findings in `current` that already appear in `baseline`. Does not remove
* or reorder `current`. `suppressed` is sorted by ruleId, then location,
* then id. Duplicate ids collapse to one record.
*/
export function compareBaselineFindings(
current: readonly unknown[],
baseline: readonly unknown[] | undefined,
): BaselineComparison {
const baselineIds = new Set<string>();
if (baseline) {
for (const finding of baseline) {
if (isIdentity(finding)) baselineIds.add(driftFindingId(finding));
}
}

const suppressed: BaselineSuppressedFinding[] = [];
const seen = new Set<string>();
for (const finding of current) {
if (!isIdentity(finding)) continue;
const id = driftFindingId(finding);
if (!baselineIds.has(id) || seen.has(id)) continue;
seen.add(id);
suppressed.push({ ruleId: finding.ruleId, location: finding.location, id });
}

suppressed.sort(compareSuppressed);
return {
compared: true,
suppressedCount: suppressed.length,
suppressed,
};
}

function compareSuppressed(a: BaselineSuppressedFinding, b: BaselineSuppressedFinding): number {
if (a.ruleId !== b.ruleId) return a.ruleId < b.ruleId ? -1 : 1;
if (a.location !== b.location) return a.location < b.location ? -1 : 1;
if (a.id !== b.id) return a.id < b.id ? -1 : 1;
return 0;
}

/** Count line shared by text and Markdown reports. */
export function baselineSuppressionSummary(count: number): string {
return `Baseline suppressions: ${count}`;
}

/** Ids recorded on a comparison, or an empty set when no baseline was compared. */
export function baselinedIdSet(comparison: BaselineComparison | undefined): Set<string> {
const ids = new Set<string>();
if (!comparison) return ids;
for (const entry of comparison.suppressed) ids.add(entry.id);
return ids;
}
15 changes: 13 additions & 2 deletions src/core-open/formatters/markdown.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
// VENDORED from @vibgrate/core-open (packages/vibgrate-core-open) by
// scripts/vendor-core-open.mjs. Do not edit here — change the source package
// and re-run the vendor script. Apache-2.0.
import type { ScanArtifact } from '../types.js';
import type { ScanArtifact, Finding } from '../types.js';
import { baselineSuppressionSummary, baselinedIdSet, driftFindingId } from '../baseline-comparison.js';
import { securityPacksLabel } from './text.js';

/** Rows shown before the infrastructure-findings table is cut with an "… N more" line. */
Expand All @@ -19,6 +20,11 @@ function formatBillable(value: number): string {
/** Generate a Markdown report from scan artifact */
export function formatMarkdown(artifact: ScanArtifact): string {
const lines: string[] = [];
const baselined = baselinedIdSet(artifact.baselineComparison);
const locationOf = (finding: Finding): string => {
const marked = baselined.has(driftFindingId(finding));
return marked ? `${finding.location} (baselined)` : finding.location;
};

// Billing (micro-project pricing) is a commercial signal attached by the full
// scan; the open base scan omits it.
Expand Down Expand Up @@ -175,7 +181,7 @@ export function formatMarkdown(artifact: ScanArtifact): string {
lines.push(`|-------|------|---------|----------|`);
for (const f of artifact.findings) {
const emoji = f.level === 'error' ? '🔴' : f.level === 'warning' ? '🟡' : '🔵';
lines.push(`| ${emoji} ${f.level} | ${f.ruleId} | ${f.message} | ${f.location} |`);
lines.push(`| ${emoji} ${f.level} | ${f.ruleId} | ${f.message} | ${locationOf(f)} |`);
}
lines.push('');
}
Expand All @@ -186,5 +192,10 @@ export function formatMarkdown(artifact: ScanArtifact): string {
lines.push('');
}

if (artifact.baselineComparison) {
lines.push(baselineSuppressionSummary(artifact.baselineComparison.suppressedCount));
lines.push('');
}

return lines.join('\n');
}
21 changes: 19 additions & 2 deletions src/core-open/formatters/sarif.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
// scripts/vendor-core-open.mjs. Do not edit here — change the source package
// and re-run the vendor script. Apache-2.0.
import type { ScanArtifact, Finding, SecurityFinding, SecuritySection, SecuritySeverity } from '../types.js';
import { driftFindingId } from '../baseline-comparison.js';

/**
* Generate a SARIF 2.1.0 document from scan artifact.
Expand All @@ -14,8 +15,9 @@ import type { ScanArtifact, Finding, SecurityFinding, SecuritySection, SecurityS
* bytes as before.
*/
export function formatSarif(artifact: ScanArtifact): object {
const suppressedIds = new Set(artifact.baselineComparison?.suppressed.map((entry) => entry.id) ?? []);
const rules = buildRules(artifact.findings);
const results = artifact.findings.map((f) => toSarifResult(f));
const results = artifact.findings.map((f) => toSarifResult(f, suppressedIds));

const runs: object[] = [
{
Expand Down Expand Up @@ -179,7 +181,8 @@ function buildRules(findings: Finding[]) {
});
}

function toSarifResult(finding: Finding) {
function toSarifResult(finding: Finding, suppressedIds: ReadonlySet<string>) {
const id = driftFindingId(finding);
return {
ruleId: finding.ruleId,
level: finding.level === 'error' ? 'error' : finding.level === 'warning' ? 'warning' : 'note',
Expand All @@ -196,5 +199,19 @@ function toSarifResult(finding: Finding) {
// Surface structured finding detail (e.g. advisory id, CVSS, fixed version)
// to consumers like GitHub code scanning without bloating the message text.
...(finding.details && Object.keys(finding.details).length > 0 ? { properties: finding.details } : {}),
// The finding stays in `results`. The suppression carries the same id as
// `baselineComparison.suppressed` so a baseline match is not a silent drop.
...(suppressedIds.has(id)
? {
suppressions: [
{
kind: 'external',
status: 'accepted',
justification: 'Matched the compared drift baseline',
properties: { id },
},
],
}
: {}),
};
}
15 changes: 13 additions & 2 deletions src/core-open/formatters/text.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
// scripts/vendor-core-open.mjs. Do not edit here — change the source package
// and re-run the vendor script. Apache-2.0.
import chalk from 'chalk';
import type { ScanArtifact, BillingSummary, ExtendedScanResults, InventoryItem, ServiceDependencyItem, ArchitectureResult, SecurityFinding, SecuritySection } from '../types.js';
import type { ScanArtifact, BillingSummary, ExtendedScanResults, InventoryItem, ServiceDependencyItem, ArchitectureResult, SecurityFinding, SecuritySection, Finding } from '../types.js';
import { baselineSuppressionSummary, baselinedIdSet, driftFindingId } from '../baseline-comparison.js';
import { driftBar } from '../ui/bar.js';
import { titleBox, panelBox } from '../ui/box.js';

Expand Down Expand Up @@ -44,6 +45,11 @@ export interface FormatTextOptions {

export function formatText(artifact: ScanArtifact, opts: FormatTextOptions = {}): string {
const lines: string[] = [];
const baselined = baselinedIdSet(artifact.baselineComparison);
const locationOf = (finding: Finding): string => {
const marked = baselined.has(driftFindingId(finding));
return marked ? `${finding.location} (baselined)` : finding.location;
};

lines.push('');
lines.push(...titleBox('Vibgrate Drift Report'));
Expand Down Expand Up @@ -94,6 +100,11 @@ export function formatText(artifact: ScanArtifact, opts: FormatTextOptions = {})
? chalk.red(`+${artifact.delta}`)
: chalk.dim('0');
lines.push(chalk.bold(' Drift Delta: ') + deltaStr + ' (vs baseline)');
}
if (artifact.baselineComparison) {
lines.push(chalk.bold(` ${baselineSuppressionSummary(artifact.baselineComparison.suppressedCount)}`));
}
if (artifact.delta !== undefined || artifact.baselineComparison) {
lines.push('');
}

Expand All @@ -117,7 +128,7 @@ export function formatText(artifact: ScanArtifact, opts: FormatTextOptions = {})
for (const f of artifact.findings) {
const icon = f.level === 'error' ? chalk.red('✖') : f.level === 'warning' ? chalk.yellow('⚠') : chalk.blue('ℹ');
lines.push(` ${icon} ${f.message}`);
lines.push(chalk.dim(` ${f.ruleId} in ${f.location}`));
lines.push(chalk.dim(` ${f.ruleId} in ${locationOf(f)}`));
}
lines.push('');
}
Expand Down
7 changes: 7 additions & 0 deletions src/core-open/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,13 @@ export {
export { formatText } from './formatters/text.js';
export { formatSarif } from './formatters/sarif.js';
export { formatMarkdown } from './formatters/markdown.js';
export {
compareBaselineFindings,
driftFindingId,
baselineSuppressionSummary,
baselinedIdSet,
} from './baseline-comparison.js';
export type { DriftFindingIdentity } from './baseline-comparison.js';

// ── Scanners (fact collection) ───────────────────────────────────────────────
export { scanNodeProjects } from './scanners/node-scanner.js';
Expand Down
7 changes: 7 additions & 0 deletions src/core-open/run-core-scan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ import { ComposerCache } from './scanners/composer-cache.js';
import { PubCache } from './scanners/pub-cache.js';
import { Semaphore } from './utils/semaphore.js';
import { computeDriftScore, generateFindings, computeProjectId, computeSolutionId } from './scoring/drift-score.js';
import { compareBaselineFindings } from './baseline-comparison.js';
import { formatText } from './formatters/text.js';
import { formatSarif } from './formatters/sarif.js';
import { formatMarkdown } from './formatters/markdown.js';
Expand Down Expand Up @@ -824,6 +825,12 @@ export async function runCoreScan(
const relBaseline = path.relative(rootDir, baselinePath);
artifact.baseline = !relBaseline || relBaseline.startsWith('..') ? path.basename(baselinePath) : relBaseline;
artifact.delta = artifact.drift.score - baseline.drift.score;
// Matched findings stay in `findings`. This block is the auditable
// record that they were already in the baseline.
artifact.baselineComparison = compareBaselineFindings(
artifact.findings,
Array.isArray(baseline.findings) ? baseline.findings : [],
);
} catch {
console.error(chalk.yellow(`Warning: Could not read baseline file: ${baselinePath}`));
}
Expand Down
40 changes: 40 additions & 0 deletions src/core-open/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -696,6 +696,35 @@ export interface BillingSummary {
billableProjects: number;
}

/**
* One current finding that was already present in the compared baseline.
* The full finding (message, level, details) stays in `findings`; this is
* the auditable identifier, not a replacement for that row.
*/
export interface BaselineSuppressedFinding {
ruleId: string;
/** Same `location` string as the finding in `findings`. */
location: string;
/**
* 32 lowercase hex characters. Derived from the finding's rule, level,
* location, and message, so the same finding always yields the same id
* and a changed message yields a different one.
*/
id: string;
}

/**
* Additive record written when `vg scan --baseline` reads a baseline file.
* Omitted entirely when no baseline was compared. `suppressed` is sorted by
* `ruleId`, then `location`, then `id`.
*/
export interface BaselineComparison {
compared: true;
/** Number of entries in `suppressed`. */
suppressedCount: number;
suppressed: BaselineSuppressedFinding[];
}

// ── Full scan artifact (stable schema) ──

export interface ScanArtifact {
Expand All @@ -709,7 +738,18 @@ export interface ScanArtifact {
solutions?: SolutionScan[];
drift: DriftScore;
findings: Finding[];
/**
* Repo-relative path of the baseline file this scan was compared against,
* or the file's basename when it lives outside the repo. Absent when
* `--baseline` was not used or the file could not be read.
*/
baseline?: string;
/**
* Which current findings were already in the baseline. Those findings stay
* in `findings`; this block is the trace that they matched. Absent when no
* baseline was compared.
*/
baselineComparison?: BaselineComparison;
delta?: number;
extended?: ExtendedScanResults;
/** Scan wall-clock duration in milliseconds */
Expand Down
2 changes: 1 addition & 1 deletion src/reporting/commands/scan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ export const scanCommand = new Command('scan')
'--fail-on <gates>',
'Fail on warn or error. architecture-finding (hard boundary violations) or architecture-warning (violations and warnings) gate on the architecture module\'s boundary findings, judged under the policy pack in force: .vibgrate/architecture.toml (policy = "hexagonal-v1" | "layered-v1" | "vertical-v1", plus any [[overlay]] rules), VIBGRATE_ARCHITECTURE_POLICY, or vg build --policy; default hexagonal-v1. The pack is named in the output. See docs/architecture-policies.md. iac-finding[=<severity>] fails on infrastructure findings from the iac-cis-v1 pack at or above <severity> (critical|high|medium|low|info; default high) and needs --iac (or --full) plus the code map — it exits 2 when the Architecture module is missing rather than passing an unevaluated tree; security-finding[=<severity>] is the umbrella across every security pack that ran. Comma-separated: at most one of warn/error/architecture-* plus any security gates, e.g. --fail-on error,iac-finding=medium',
)
.option('--baseline <file>', 'Compare against baseline')
.option('--baseline <file>', 'Compare against a baseline and record matched findings')
.option('--changed-only', 'Only scan changed files')
.option(
'-e, --exclude <glob>',
Expand Down
Loading
Loading