Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
88f6839
Build/Test Tools: Generate PHPStan baselines for more than one config…
westonruter Nov 16, 2026
1d13e5b
Build/Test Tools: Hold selected paths to PHPStan rule level 10.
westonruter Nov 16, 2026
e1d41e3
Build/Test Tools: Run both PHPStan analyses from `composer phpstan`.
westonruter Nov 16, 2026
772bd56
Build/Test Tools: Run the level 10 PHPStan analysis in CI.
westonruter Nov 16, 2026
e81d6a5
Build/Test Tools: Analyze only the strict paths at PHPStan level 10.
westonruter Nov 16, 2026
e3767eb
Build/Test Tools: Document the level 10 PHPStan analysis.
westonruter Nov 16, 2026
d444fe3
Build/Test Tools: Run a single PHPStan analysis for single-analysis a…
westonruter Nov 16, 2026
79bd33f
Build/Test Tools: Write a local PHPStan override's baselines to its o…
westonruter Nov 18, 2026
a2a08ff
Build/Test Tools: Point to the right job for PHPStan errors left out …
westonruter Nov 16, 2026
af6e512
Build/Test Tools: Clarify running one PHPStan analysis and adding a s…
westonruter Nov 16, 2026
d02ce97
Build/Test Tools: Stop holding the reverted Secrets API to PHPStan le…
westonruter Nov 18, 2026
aec7d48
Build/Test Tools: Stop naming the Secrets API among the level 10 paths.
westonruter Nov 18, 2026
71f3ec7
Build/Test Tools: Include the level 10 PHPStan baselines through a lo…
westonruter Nov 18, 2026
d3bfc6b
Build/Test Tools: Write a PHPStan baselines loader before the analysis.
westonruter Nov 18, 2026
8648fd9
Build/Test Tools: Name the PHPStan analyses default and strict.
westonruter Nov 18, 2026
618a81b
Build/Test Tools: Pass the encoding of PHPStan results as an option.
westonruter Nov 18, 2026
702a2fb
Build/Test Tools: Split the sentence regenerating a PHPStan baseline.
westonruter Nov 18, 2026
8ecdd5d
Build/Test Tools: Name the configuration including the other in the P…
westonruter Nov 18, 2026
7bdd35b
Build/Test Tools: Quote the paths in the command regenerating a PHPSt…
westonruter Nov 18, 2026
0d9d9ca
Build/Test Tools: Rewrap a comment in the PHPStan baseline generator.
westonruter Nov 18, 2026
628ded6
Build/Test Tools: Skip an option's value without index arithmetic in …
westonruter Nov 18, 2026
ff28f2d
Build/Test Tools: Check whether PHPStan could dump a configuration's …
westonruter Nov 18, 2026
20f269a
Build/Test Tools: Allow any whitespace after the key of a dumped PHPS…
westonruter Nov 18, 2026
3e927df
Build/Test Tools: Run PHPStan's `analyse` command through a function …
westonruter Nov 18, 2026
e069ef2
Build/Test Tools: Unpack the arguments after PHPStan's `analyse` comm…
westonruter Nov 18, 2026
6428c61
Build/Test Tools: Run PHPStan on pull requests changing only the stri…
westonruter Nov 18, 2026
aff84be
Build/Test Tools: Lift Composer's timeout for each single PHPStan ana…
westonruter Nov 18, 2026
246e147
Build/Test Tools: Name the configuration in every command regeneratin…
westonruter Nov 18, 2026
c5bc433
Build/Test Tools: Say what skipping the strict PHPStan analysis assumes.
westonruter Nov 18, 2026
e1348ea
Build/Test Tools: Combine the files changed in the base branch from b…
westonruter Nov 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 90 additions & 10 deletions .github/workflows/phpstan-pull-request-comments.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,20 @@ jobs:
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ github.token }}

# The results of the analysis holding selected paths to a stricter rule level. A pull request whose branch does
# not have that analysis yet uploads none, which is fine. Whether they were expected is decided from the results
# above.
- name: Download the results of the strict analysis
id: download-strict
if: ${{ steps.download.outcome == 'success' }}
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
continue-on-error: true
with:
name: phpstan-results-strict
path: ${{ runner.temp }}/phpstan-results-strict
run-id: ${{ github.event.workflow_run.id }}
github-token: ${{ github.token }}

- name: Create, update, or resolve the PHPStan comment
if: ${{ steps.download.outcome == 'success' }}
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
Expand All @@ -74,7 +88,7 @@ jobs:
// anything at all. They are therefore checked for shape, and only accepted for the pull request whose
// current head is the commit that the run analyzed. That also discards the results of a run that was
// overtaken by a newer push.
const results = JSON.parse( fs.readFileSync( `${ process.env.RUNNER_TEMP }/phpstan-results/results.json`, 'utf8' ) );
const results = JSON.parse( fs.readFileSync( `${ process.env.RUNNER_TEMP }/phpstan-results/results.json`, { encoding: 'utf8' } ) );
const issue_number = results.pull_request;
if ( ! Number.isSafeInteger( issue_number ) || issue_number <= 0 || ! Array.isArray( results.errors ) ) {
core.setFailed( 'The analysis results are malformed.' );
Expand All @@ -93,12 +107,62 @@ jobs:

const asString = ( value ) => ( typeof value === 'string' ? value : null );
const asSha = ( value ) => ( typeof value === 'string' && /^[0-9a-f]{40}$/.test( value ) ? value : null );
// A rule level is a number, or `max`. Results from a branch that does not report the level have none.
const asLevel = ( value ) => ( ( Number.isSafeInteger( value ) && value >= 0 ) || value === 'max' ? value : null );
const normalize = ( error ) => ( {
file: asString( error.file ),
line: Number.isSafeInteger( error.line ) && error.line > 0 ? error.line : null,
message: ( asString( error.message ) ?? '' ).slice( 0, 1000 ),
identifier: /^[A-Za-z0-9.]+$/.test( error.identifier ) ? error.identifier : null,
ignorable: error.ignorable === true,
// The analysis that reported the error, and the rule level it analyzed at. Neither is reported per error:
// they are those of the results the error was read from, and are kept in the history from there.
analysis: error.analysis === 'strict' ? 'strict' : 'default',
level: asLevel( error.level ),
} );

// The paths held to a stricter rule level are analyzed a second time, and that analysis uploads its results
// separately. The results above list the analyses the run made, and when the strict one is among them but
// its results are missing, the comment is left as it is: every error that only analysis reported would
// otherwise be shown as resolved. A run from a branch that does not have that analysis yet lists none, and
// has only these results.
const expectsStrict = Array.isArray( results.analyses ) && results.analyses.includes( 'strict' );
let strictErrors = [];
let strictLevel = null;
let strictChangedInBase = [];
if ( expectsStrict ) {
let strictResults = null;
try {
strictResults = JSON.parse( fs.readFileSync( `${ process.env.RUNNER_TEMP }/phpstan-results-strict/results.json`, { encoding: 'utf8' } ) );
} catch ( error ) {
core.info( `The results of the strict analysis are missing, so the comment is left as it is: ${ error.message }` );
return;
}
if ( strictResults?.pull_request !== issue_number || ! Array.isArray( strictResults.errors ) ) {
core.setFailed( 'The results of the strict analysis are malformed.' );
return;
}
strictLevel = asLevel( strictResults.level );
strictErrors = strictResults.errors.map( ( error ) => normalize( { ...error, analysis: 'strict', level: strictLevel } ) );
strictChangedInBase = Array.isArray( strictResults.changed_in_base ) ? strictResults.changed_in_base : [];
}

// The strict analysis includes everything the default one does, so an error it reports at a lower level is
// reported by both. Such an error is listed once, as found by the analysis of the whole codebase, and the
// strict one only adds what is left over. Errors are matched as they are below, apart from their line.
const errorKey = ( error ) => JSON.stringify( [ error.file, error.identifier, error.message ] );
const defaultCounts = new Map();
const defaultErrors = results.errors.map( ( error ) => normalize( { ...error, analysis: 'default', level: results.level } ) );
for ( const error of defaultErrors ) {
defaultCounts.set( errorKey( error ), ( defaultCounts.get( errorKey( error ) ) ?? 0 ) + 1 );
}
const strictOnly = strictErrors.filter( ( error ) => {
const count = defaultCounts.get( errorKey( error ) ) ?? 0;
if ( count > 0 ) {
defaultCounts.set( errorKey( error ), count - 1 );
return false;
}
return true;
} );

// An error is recognized from one run to the next by its file, identifier and message, but not its line,
Expand All @@ -107,10 +171,10 @@ jobs:
// is fixed or added, those after it shift to the history of the one before or after, so their first reports
// can be off, and the one shown as resolved or new is the last rather than the one that was. That is rarer
// than lines being added or removed above them, which would throw off matching them by line.
const errors = results.errors.map( normalize );
const errors = [ ...defaultErrors, ...strictOnly ];
Comment thread
westonruter marked this conversation as resolved.
Resolved
const occurrences = new Map();
for ( const error of [ ...errors ].sort( ( a, b ) => ( a.line ?? 0 ) - ( b.line ?? 0 ) ) ) {
const key = JSON.stringify( [ error.file, error.identifier, error.message ] );
const key = errorKey( error );
const occurrence = occurrences.get( key ) ?? 0;
occurrences.set( key, occurrence + 1 );
error.key = `${ key }#${ occurrence }`;
Expand Down Expand Up @@ -164,7 +228,11 @@ jobs:

// Files the base branch has changed since the pull request branched from it, in which the line numbers
// reported can be off in the head commit, since the analysis ran on the head commit merged into the base.
const changedInBase = new Set( Array.isArray( results.changed_in_base ) ? results.changed_in_base.filter( ( file ) => typeof file === 'string' ) : [] );
// Each analysis only lists those among the files it reported errors in, so the lists of both are combined.
const changedInBase = new Set(
[ ...( Array.isArray( results.changed_in_base ) ? results.changed_in_base : [] ), ...strictChangedInBase ]
.filter( ( file ) => typeof file === 'string' )
);
const lineMayBeOff = ( error ) => ! error.fixed && Boolean( error.line ) && changedInBase.has( error.file );
const filesWithLinesOff = new Set( active.filter( lineMayBeOff ).map( ( error ) => error.file ) );

Expand Down Expand Up @@ -206,7 +274,10 @@ jobs:

// GitHub shows only the first few annotations of each kind from a step on the page of the workflow run, so
// errors left out of a table are pointed to in the log of the step that reported them, which has them all.
const where = `the log of the [workflow run](${ run.html_url }), in the “Report the analysis results” step`;
// Each analysis runs in a job of its own, and an error found only by the strict one is only in that job's log.
const where = strictOnly.length > 0
? `the log of the [workflow run](${ run.html_url }), in the “Report the analysis results” step of each of its jobs (errors in strict paths are only in the “strict paths” job)`
: `the log of the [workflow run](${ run.html_url }), in the “Report the analysis results” step`;
const table = ( rows, rowLimit, { withResolution = false } = {} ) => {
const shown = rows.slice( 0, rowLimit );
if ( shown.length === 0 ) {
Expand All @@ -215,10 +286,14 @@ jobs:
: `The ${ rows.length } errors are not listed here, to keep this comment short. They are listed in ${ where }.`;
}
const more = `…and ${ rows.length - shown.length } more, which are listed in ${ where }.`;
// The rule level is only shown in a table listing an error from the strict analysis, so that one listing
// only errors from the analysis of the whole codebase looks as it did before there were two. A level
// that was not reported, as by a branch from before it was, is left blank.
const withLevel = shown.some( ( error ) => error.analysis === 'strict' );
const lines = [
`| Location | Error | Identifier | First reported |${ withResolution ? ' Resolved |' : '' }`,
`| --- | --- | --- | --- |${ withResolution ? ' --- |' : '' }`,
...shown.map( ( error ) => `| ${ location( error ) } | ${ code( error.message ) } | ${ identifier( error ) } | ${ commit( error.introduced ) } |${ withResolution ? ` ${ commit( error.fixed ) } |` : '' }` ),
`| Location | Error | Identifier |${ withLevel ? ' Level |' : '' } First reported |${ withResolution ? ' Resolved |' : '' }`,
`| --- | --- | --- |${ withLevel ? ' --- |' : '' } --- |${ withResolution ? ' --- |' : '' }`,
...shown.map( ( error ) => `| ${ location( error ) } | ${ code( error.message ) } | ${ identifier( error ) } |${ withLevel ? ` ${ error.level ?? '' } |` : '' } ${ commit( error.introduced ) } |${ withResolution ? ` ${ commit( error.fixed ) } |` : '' }` ),
];
if ( shown.length < rows.length ) {
lines.push( '', more );
Expand Down Expand Up @@ -248,6 +323,11 @@ jobs:
`> ${ filesWithLinesOff.size === 1 ? 'A file with errors has' : `${ filesWithLinesOff.size } files with errors have` } changed in ${ code( pr.base.ref ) } since this branch was last updated, so the line numbers reported in ${ filesWithLinesOff.size === 1 ? 'it' : 'them' } may be off and are not linked. Merge the latest ${ code( pr.base.ref ) } into this branch for accurate links.`,
].join( '\n' ) );
}
if ( active.some( ( error ) => error.analysis === 'strict' ) ) {
const strictError = strictLevel === null ? 'An error found by the strict analysis' : `An error at level ${ strictLevel }`;
const stricterLevel = strictLevel === null ? 'a stricter rule level' : `rule level ${ strictLevel }`;
sections.push( `${ strictError } is in a path that ${ code( 'phpstan-strict.neon.dist' ) } holds to ${ stricterLevel }, which the rest of the codebase is not yet held to. See [Stricter analysis of selected paths](${ readmeUrl }#stricter-analysis-of-selected-paths).` );
}
} else {
// Resolved by the last commit that resolved anything, so that later commits do not change the comment.
const resolvedBy = resolved.at( -1 )?.fixed ?? run.head_sha;
Expand Down Expand Up @@ -316,8 +396,8 @@ jobs:

// The errors tracked are taken in the order they are listed, so that each one listed keeps its history.
const tracked = [ ...unignorable, ...ignorable ].slice( 0, trackedLimit ).concat( resolved ).map( ( error ) => {
const { key, file, line, message, identifier, ignorable, introduced, seen, fixed } = error;
return { key, file, line: lineMayBeOff( error ) ? null : line, message, identifier, ignorable, introduced, seen, fixed };
const { key, file, line, message, identifier, ignorable, analysis, level, introduced, seen, fixed } = error;
return { key, file, line: lineMayBeOff( error ) ? null : line, message, identifier, ignorable, analysis, level, introduced, seen, fixed };
} );
sections.push( `<!-- phpstan-history:${ zlib.deflateRawSync( JSON.stringify( tracked ) ).toString( 'base64' ) } -->` );

Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/phpstan-static-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,10 @@ on:
- 'composer.*'
# These files configure PHPStan. Changes could affect the outcome.
- 'phpstan.neon.dist'
- 'phpstan-strict.neon.dist'
- 'tests/phpstan/base.neon'
- 'tests/phpstan/baselines/**'
- 'tests/phpstan/baselines-strict/**'
# Confirm any changes to relevant workflow files.
- '.github/workflows/phpstan-static-analysis.yml'
- '.github/workflows/reusable-phpstan-static-analysis-v1.yml'
Expand Down
Loading
Loading