Skip to content

Build/Test Tools: Include the PHPStan baselines through a PHP loader - #16049

Closed
westonruter wants to merge 39 commits into
WordPress:trunkfrom
westonruter:phpstan-baseline-loader
Closed

westonruter wants to merge 39 commits into
WordPress:trunkfrom
westonruter:phpstan-baseline-loader

Conversation

@westonruter

@westonruter westonruter commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

✅ Committed in r65566 (19379c8).


phpstan.neon.dist currently lists every per-identifier baseline in its includes, between # phpstan:baselines markers, and composer phpstan:baselines rewrites that list whenever a baseline is added or deleted. Every PR that fixes the last error of some identifier therefore edits the same list, so two PRs retiring different baselines conflict with each other, and whichever merges second has to be rebased.

This replaces the list with a single include of tests/phpstan/baselines/loader.php, a PHP config file that returns the baselines in its directory as its includes. NEON can only include files by name, but a PHP config can find them itself. A baseline now takes effect when it is added and stops when it is deleted, with no edit of the configuration. PHPStan keys its config cache on the included files, so the change is picked up on the next run.

Changes

  • tests/phpstan/baseline-includes.php (new) holds what the loaders and the generator share:
    • find_baselines() lists the files in a directory named for an error identifier, matched with the pattern PHPStan validates identifiers with, followed by .neon. A stray file such as a merge tool's argument.type_BACKUP_1234.neon is neither loaded nor retired. A directory that cannot be listed throws, rather than silently loading no baselines.
    • get_baseline_includes() returns those as bare file names, which PHPStan resolves against the loader's path. Absolute paths built from __DIR__ would be real paths, which break a checkout reached through a symbolic link.
  • tests/phpstan/baselines/loader.php (new) is a two-line stub that requires the shared file and returns get_baseline_includes( __DIR__ ). The generator writes it, and rewrites it on every run, so it cannot drift.
  • phpstan.neon.dist includes the loader in place of the 31 listed baselines.
  • tests/phpstan/generate-baselines.php:
    • The markers, update_config_includes(), find_includes_end(), and strip_baseline_includes() are removed.
    • While regenerating, the generator sets WP_PHPSTAN_GENERATING_BASELINES to the real path of the output directory, and the loader in that directory includes nothing. Previously only lines in the top-level config were stripped, so composer phpstan:baselines -- --config=phpstan.neon, with a local phpstan.neon that includes phpstan.neon.dist as the README suggests, left every baseline active and then deleted those it fully covered. This now works however the loader is included.
    • Baselines in any other directory stay in force, so each error is baselined once. A stricter configuration that includes phpstan.neon.dist and keeps its own baselines, such as one holding selected paths to rule level 10, gets only the errors the default baselines leave. The generator says so when the configuration or the output directory is not the default. An inherited value of the variable is cleared first.
    • With --output-dir, a directory without a loader gets one, with a note that the configuration has to include it.
    • Each baseline's header gives the command to regenerate it, which now includes --config and --output-dir when they were not the defaults.
    • An analysis that reports no errors, the usual result for a stricter configuration's own directory, is written as no baselines rather than rejected.
    • --identifier values are checked to be identifiers, so one such as ../../../phpstan cannot name a file outside the baselines directory to be retired, and a baseline that cannot be written stops the run.
  • Baseline headers no longer say to delete the includes entry along with an emptied file, and say that regenerating suppresses the baselines in that directory.
  • tests/phpstan/README.md and the failure message of the PHPStan workflow describe the loader in place of the markers.

Testing

With PHPStan 2.3.6:

  • composer phpstan -- --configuration=phpstan.neon.dist reports no errors. Since reportUnmatchedIgnoredErrors is enabled, every baseline entry still matches.
  • composer phpstan:baselines and npm run typecheck:php:baselines both report the same 983 errors and leave the working tree clean, the loader included.
  • composer phpstan:fix, after raising the count of one baseline entry by one, lowers it back through the loader, prunes nothing, and ends with no errors. --prune deletes a baseline left with ignoreErrors: [].
  • A level 10 configuration including phpstan.neon.dist, generated with --config and --output-dir into a directory of its own, gets 15 baselines plus a loader. None of them repeats an entry of the default baselines. Analyzing with both loaders included reports no errors, and running the command from a header regenerates identical files.
  • Analyzing through a symbolically linked path to the checkout applies every baseline.

This came out of a prototype adding glob patterns to includes in PHPStan itself (phpstan/phpstan-src#7099), which turned out not to be needed, since a PHP config can do the same with the PHPStan release core already uses.

Trac ticket: https://core.trac.wordpress.org/ticket/65817

Use of AI Tools

AI assistance: Yes
Tool(s): Claude Code
Model(s): Claude Opus 5.5 (medium and high effort)
Used for: Investigating PHPStan's handling of includes, implementing the loader and the generator changes, code review, testing, and drafting this description. All of it was directed and reviewed by me.


This Pull Request is for code review only. Please keep all other discussion in the Trac ticket. Do not merge this Pull Request. See GitHub Pull Requests for Code Review in the Core Handbook for more details.

westonruter and others added 5 commits November 16, 2026 18:09
`phpstan.neon.dist` listed every per-identifier baseline in its
`includes`, between `# phpstan:baselines` markers, and
`generate-baselines.php` rewrote that list whenever a baseline was added
or deleted. Every PR that retired a baseline therefore edited the same
list, so two such PRs conflicted with each other.

The config now includes `tests/phpstan/baselines/loader.php`, a PHP
config file that returns every `.neon` file in its directory as its
`includes`. NEON can only include files by name, but a PHP config can
find them itself. A baseline takes effect when it is added and stops
when it is deleted, with no edit of the config. PHPStan keys its config
cache on the included files, so the next run sees the change. The
resolved `ignoreErrors` are identical to those from the explicit list.

The generator no longer maintains the list: the markers,
`update_config_includes()` and `find_includes_end()` are removed, and so
is the line in each baseline's header about deleting its `includes`
entry. While regenerating, it still suppresses the baselines by
stripping includes from a temporary copy of the config: the loader
itself, and anything else inside the output directory, so `--output-dir`
still yields a complete baseline rather than a differential one.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 020072d9-40ca-4276-af5c-1783ac89d13b
The generator suppressed the baselines while regenerating them by
dropping the loader's line from a copy of the config. That only worked
for one spelling of one line in the top-level config:

* A quoted path, a trailing comment, or a `%rootDir%` path left the loader
  in place.
* `--config=phpstan.neon`, with that file including `phpstan.neon.dist`
  as the README suggests, left it in place too, since only the top-level
  config is rewritten.

Either way every baseline stayed active, so every identifier fully
covered by one reported nothing, and the run deleted those baselines.

The loader now includes nothing when WP_PHPSTAN_GENERATING_BASELINES is
set, and the generator sets it for the PHPStan process. This works
however and wherever the loader is included, and the separate
`BASELINES_LOADER` path, which repeated the default output directory, is
no longer needed. Includes inside the output directory are still dropped
from the copy, for a loader written to a directory other than the
default.

The docblock of `strip_baseline_includes()` no longer describes a
baseline included by hand alongside the loader. With the default output
directory that includes the file twice, which PHPStan rejects.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 020072d9-40ca-4276-af5c-1783ac89d13b
…includes.

`strip_baseline_includes()` normalized the output directory again for
every line of the config's `includes`, though it never changes.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 020072d9-40ca-4276-af5c-1783ac89d13b
With the marker list gone, nothing included the baselines written by
`composer phpstan:baselines -- --output-dir=<path>`. The default loader
only includes its own directory, so they were written and then ignored,
with no hint why.

When the output directory has no `loader.php`, the generator now copies
the default one there and says that the config has to include it. A
later run against that directory finds its loader inside the output
directory, which is dropped from the config while it regenerates.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 020072d9-40ca-4276-af5c-1783ac89d13b
The loader included every `.neon` file in its directory, so a stray one,
such as `argument.type_BACKUP_1234.neon` left by a merge tool, was loaded
along with the real baseline. Its entries then matched nothing, failing
the analysis with ignore.unmatched errors that cannot be ignored. Before
the loader, only the files listed by name were loaded.

The loader now only includes files whose name is an error identifier,
matched with the pattern PHPStan validates identifiers with, followed by
`.neon`. Every baseline the generator writes is named that way.
`find_baselines()` matches the same names, so a stray file is not
retired or pruned either.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 020072d9-40ca-4276-af5c-1783ac89d13b
westonruter and others added 4 commits November 16, 2026 20:05
…onfiguration.

Every loader includes nothing while WP_PHPSTAN_GENERATING_BASELINES is
set, including the copy written to a directory given with `--output-dir`,
so the baselines are already out of the way during the analysis. What
`strip_baseline_includes()` still removed was a baseline listed by name
inside the output directory, which in the default directory PHPStan
rejects as a duplicate of the loader's include, and elsewhere is no longer
a supported setup.

The wrapper setting `tmpDir` now includes the configuration directly, so
the stripped copy of it, and the function producing it, are removed.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 020072d9-40ca-4276-af5c-1783ac89d13b
The message shown when the PHPStan workflow fails still said that
`composer phpstan:fix` and `composer phpstan:baselines` update the list of
baselines in `phpstan.neon.dist`. There is no such list any more, since
the loader finds the baselines itself.

The README said the loader includes every `.neon` file in its directory,
but it only includes those named for an error identifier.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
The loader included no baselines whenever WP_PHPSTAN_GENERATING_BASELINES
was set, to any value. Exporting it as `0` or as an empty string to turn
it off, or leaving it set in a shell, then silently dropped every baseline
from an ordinary run, which reported thousands of errors with nothing
pointing at the cause.

The loader now only does so when the variable is `1`, the value the
generator sets.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…n copied.

`composer phpstan:baselines --output-dir` copies the loader into the
directory it writes to, docblock included. The docblock said that
`phpstan.neon.dist` includes the file, which only holds for the default
directory, and a reader of a copy could take the baselines there to be in
force already. It now refers to whichever configuration includes the
file, and notes that the file is copied.

The docblock of GENERATING_BASELINES_ENV in the generator is also
rewrapped, after the previous commit added a sentence to it.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
@westonruter
westonruter marked this pull request as ready for review November 17, 2026 07:44
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs.

Core Committers: Use this line as a base for the props when committing in SVN:

Props westonruter, apermo.

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.


Props Bot commands

You can trigger Props Bot actions by commenting on this PR:

  • @propsbot add <username> to give props to a GitHub or WordPress.org user. Requires write access.
  • @propsbot remove <username> to leave a user out of the props. Requires write access.
  • @propsbot refresh to refresh this list. Adding the props-bot label works too.

@westonruter

Copy link
Copy Markdown
Member Author

@apermo this is a follow up to simplify the identifier-specific baselines which was uncovered when exploring phpstan/phpstan-src#7099. It will next then feed into #16021 to start enforcing rule level on a subset of the project. Please take a look!

@apermo apermo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (besides the yoda conditions... but they are coding standards, so all fine)

westonruter and others added 16 commits November 17, 2026 11:03
The generator set WP_PHPSTAN_GENERATING_BASELINES to 1, and every loader
then included nothing. A stricter configuration that includes
`phpstan.neon.dist` and keeps its own baselines in another directory, such
as one holding selected paths to level 10, means for those to hold only
the errors the default baselines leave, so that each error is baselined
once. Regenerating them with every loader off wrote a duplicate of each
default entry for those paths. The analysis then matched every error with
the first of the two, and reported the other as unmatched, which cannot
be ignored.

The generator now sets the variable to the resolved output directory, and
a loader only includes nothing when it names its own directory. The
baselines elsewhere stay in force, so that only the errors they leave are
written. Regenerating the default directory is unchanged, including
through a local `phpstan.neon` that includes `phpstan.neon.dist`, and a
value left set by mistake still matches no directory.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
When the analysis reports no errors, PHPStan writes the baseline as
`return [];`, without the `parameters` key it otherwise has. The generator
rejected that as an unexpected structure and stopped before writing
anything.

That used to need a codebase free of errors, but it is now the usual
outcome for a directory whose errors the baselines elsewhere already
cover, such as with `--output-dir` and the default configuration. It is
now read as no entries.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…ine generator.

The two previous commits each removed a space before the `=` of the
`$parameters` assignment in `read_baseline()` by accident, leaving it out
of line with the `$ignore_errors` assignment below it.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
The loader built each baseline's path from __DIR__, which PHP resolves to
the real path, following any symbolic link. When the checkout was reached
through a link, the baselines were then included from the real path, and
the paths their entries name relative to themselves resolved there too,
while the analyzed files were under the linked path. No entry matched, so
every baselined error was reported, along with every entry as unmatched.
Listing the baselines one by one, as before the loader, did not have this
problem, since those paths were relative to the configuration.

The loader now returns bare file names, which PHPStan resolves against the
loader's path as the configuration reached it.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
The loader held the logic for finding the baselines, and the generator
copied it word for word into any other directory given with
`--output-dir`. A later fix to the loader never reached a copy, since one
is only written where there is no loader yet. The pattern for a baseline's
file name was also written twice, in the loader and in the generator,
with a comment asking that the two be kept in sync.

That logic now lives in `tests/phpstan/baseline-includes.php`, together
with the pattern, the environment variable's name, and find_baselines(),
which the generator requires. A loader is a stub that requires it and
returns get_baseline_includes() for its own directory. The generator
writes such a stub for a directory given with `--output-dir`, with the
path to the shared file relative to that directory, and the default
directory's loader is the stub it builds for that directory.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…e regenerating.

Writing baselines anywhere but the default directory leaves the default
baselines in force, so only the errors they leave are written. A run
expecting every error came up short with nothing to say why. The
generator now says so when the output directory is not the default.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…equire path.

build_loader() worked out the path from a loader to the shared
`baseline-includes.php` starting at the output directory as given. The
loader follows it from its __DIR__, which PHP resolves to the real path.
When the output directory was reached through a symbolic link, the two
disagreed, and the loader required a file that does not exist, so PHPStan
failed as soon as the configuration included it.

The path is now worked out between the resolved directories.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…on-default run.

The note that baselines in other directories stay in force while
regenerating was only printed for an output directory other than the
default. They also stay in force when the default directory is
regenerated with a configuration that includes another directory's
loader, such as a local `phpstan.neon` adding stricter baselines of its
own. The default baselines were then written without the errors those
cover, with nothing said, and CI, which does not include that loader,
reported them.

The note is now printed unless both the configuration and the output
directory are the defaults, with the directories compared as resolved,
as the loader compares them.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…eration.

A loader was only written to a directory that had none, so the committed
`tests/phpstan/baselines/loader.php`, and a loader written to any other
directory, were copies of what build_loader() makes that could drift from
it unnoticed: a change to one and not the other kept both working, but
differently.

The generator now writes the loader on every run where it differs, so any
drift shows up as a change to the loader. It only asks for the loader to
be included when the directory had none before. The loader says it is
written by the generator and not to be edited by hand.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…obbing it.

find_baselines() passed the directory to glob() unescaped, so a checkout
whose path holds a glob metacharacter, such as the brackets of
`wp [old]/`, matched nothing. The loader then included no baselines, and
every baselined error was reported, with nothing pointing at why. A
glob() failure was likewise read as no baselines.

The directory is now listed with scandir(), and one that cannot be listed
throws, which stops PHPStan while it loads the configuration rather than
dropping every suppression. find_baselines() returns the file names it
finds, which is what the loader includes, rather than absolute paths it
turned back into names. The generator adds the directory where it needs a
path, and `--prune` of a directory not created yet finds nothing to prune,
as before.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…ot an identifier.

The value of `--identifier` names the file a baseline is written to and
retired from, and it was not checked. One such as `../../../phpstan`
named a file outside the baselines directory, which a run deleted when no
error with that identifier was reported, taking a local `phpstan.neon`
with it.

An identifier is now checked against the same pattern a baseline's file
name is, and the run stops on one that does not match.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…ines generator.

An earlier edit left its lines at about 45 columns, where the rest of the
usage message wraps at about 80.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…an baselines loaders.

When the output directory did not exist yet, WP_PHPSTAN_GENERATING_BASELINES
was set to its unresolved path instead. No loader can be there to match
it, so it did nothing. It is now only set when the directory resolves.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…not spelling.

A loader included nothing when WP_PHPSTAN_GENERATING_BASELINES was the
same string as its __DIR__. The generator set it to the realpath() of the
output directory, which resolves symbolic links but keeps the case it was
given. On a case-insensitive file system, `--output-dir` spelled as
`Tests/PHPStan/Baselines` therefore missed the loader of
`tests/phpstan/baselines`: the baselines stayed in force, their errors
were not reported, and the run retired every one of them.

Directories are now compared with is_same_directory(), by the device and
inode the file system reports, falling back to the resolved paths where
it reports no inode. The loader, the note about other baselines staying
in force, and build_loader() all use it. build_loader() finds the common
ancestor of the output directory and the shared file that way, so the
loader it writes no longer changes with the spelling of the option.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…before regenerating.

The generator only set the variable when the output directory existed,
and otherwise left alone any value inherited from the shell. One exported
to name the default directory, while writing to a new one, then switched
off the default baselines during the analysis, and the new directory got
a duplicate of them rather than only the errors they leave.

The variable is now cleared first, and set only when the output directory
exists.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
westonruter and others added 13 commits November 17, 2026 13:09
…thout a stack trace.

find_baselines() throws when the directory cannot be listed, which is
what stops PHPStan when a loader calls it. In the generator the exception
went uncaught, so the run ended in a fatal error and a stack trace, where
every other failure prints a message and exits with 1.

The generator now lists the baselines through list_baselines(), which
reports the failure that way.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
A full regeneration retired every `.neon` file in the output directory
whose name passes for an identifier when that identifier reported
nothing, and `--prune` deleted any such file with an empty
`ignoreErrors`. Pointed at a directory holding other configuration, that
meant `--output-dir=tests/phpstan` deleted `base.neon`, and
`--output-dir=.` the root `phpstan.neon`.

Both now only delete a file that starts with the header the generator
writes for its identifier, and leave any other in place with a warning.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
The generator did not check whether writing each baseline succeeded. On a
read-only file or a full disk it still printed the file's counts as if it
had been written, went on to retire others, and exited with 0, leaving
the stale baseline in place.

It now stops with an error naming the file, as it already does for the
loader.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…eration does.

Each baseline said that regenerating it reruns the analysis with that
file suppressed. The loader suppresses every baseline in its directory
while the generator runs, even with `--identifier`, and the generator then
writes only the named one. The header now says so, and the baselines are
regenerated to carry it.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
… other configuration.

Retiring and pruning already leave alone a file the generator did not
write, but a directory holding one was still written to. Its loader would
include such a file as a baseline, such as `base.neon` with
`--output-dir=tests/phpstan`, and a reported identifier of the same name
would overwrite it.

The generator now stops before any analysis when the output directory
holds a `.neon` file named like a baseline that does not start with the
header it writes, naming the files and asking for a directory of its own.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…der's path.

On PHP before 8, substr() returns false rather than an empty string when
the start is at the end of the string, so PHPStan at level 10 reported
passing its result to ltrim(). That cannot happen here, since the shared
file's path is always longer than that of a directory holding it, so the
result is cast.

Co-Authored-By: Claude Opus 5.5 (medium effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…e endings.

is_generated_baseline() compared the start of the file with the header's
first line including its LF. In a checkout that converts line endings to
CRLF, as Git does on Windows with `core.autocrlf`, every committed
baseline then failed the check: the generator refused to run on the
default directory as one holding files it did not write, and `--prune`
left every emptied baseline in place.

Only the first line is now read, and compared without its line ending,
which also spares reading each baseline in full to compare one line.

Co-Authored-By: Claude Opus 5.5 (high effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…nes generator did not write.

The check for files the generator did not write only looked at `.neon`
files named like baselines. An output directory holding some other
`loader.php`, such as another tool's bootstrap, passed it, and the
generator then replaced that file with its own loader.

The run now also stops before any analysis when the output directory
holds a `loader.php` without the statement every generated loader ends
in. That statement is kept in LOADER_RETURN, which build_loader() writes
and is_generated_loader() looks for.

Co-Authored-By: Claude Opus 5.5 (high effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…aseline's regenerate command.

Each baseline's header gives the command that regenerates it, which only
named the identifier. For baselines generated with a configuration or an
output directory other than the default, following it regenerated the
default directory instead: the default baseline was rewritten as only the
errors the other directory's baselines leave, and the one the header
belonged to was not touched.

The command now carries `--config` and `--output-dir` where the run used
other than the defaults, so the default baselines are unchanged. The
header's first line still names only the identifier, which is what
is_generated_baseline() recognizes.

Co-Authored-By: Claude Opus 5.5 (high effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
…e very end.

BASELINE_FILE_PATTERN ended in `$`, which without the `D` modifier also
matches before a final newline, so a file named `argument.type.neon`
followed by a newline was taken for a baseline. It now ends in `\z`.

Co-Authored-By: Claude Opus 5.5 (high effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
… check.

A run now stops before its analysis when the output directory holds a
file named like a baseline that the generator did not write, so the
retire loop's check that each file it deletes is one the generator wrote
could no longer find one. It is removed, and is_generated_baseline()
says where it is used instead.

Co-Authored-By: Claude Opus 5.5 (high effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
The baselines tooling had grown guards against ways of running it that
the workflows it serves never do, all of them only reachable by pointing
it at the wrong place, and all of them recoverable from version control:

* is_same_directory(), comparing directories by inode so that an
  `--output-dir` typed in the wrong case on a case-insensitive file
  system still matched the loader. The loader now compares the real path
  it is given with its __DIR__, and build_loader() works out its require
  path with get_relative_path() between the real paths, rather than by
  walking the ancestors of both.
* The check before the analysis that the output directory holds no
  `.neon` file or `loader.php` the generator did not write, along with
  is_generated_baseline(), its handling of CRLF line endings,
  is_generated_loader(), LOADER_RETURN, and `--prune`'s own header check.
* quote_shell_argument(), for the regenerate command in a header, whose
  options are repository-relative paths.
* list_baselines(), which turned find_baselines()'s exception into a
  message. The uncaught exception carries the same message.

Co-Authored-By: Claude Opus 5.5 (high effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a
The generator's file docblock still said it analyzes with the existing
baselines suppressed. Only those being regenerated are; the rest stay in
force.

Co-Authored-By: Claude Opus 5.5 (high effort) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnky3HeNQTYTAish2dfgd9
Claude-Code-Session-Id: 7fbbd88f-e571-4959-a07e-802b483fab5a

@westonruter westonruter left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOL. My review rounds got a bit out of hand. I reined in Claude at the end and had it undo a bunch of unnecessary stuff.

wporg-sync pushed a commit that referenced this pull request Nov 17, 2026
The per-identifier baselines were listed one by one in `phpstan.neon.dist`, between markers that `composer phpstan:baselines` rewrote whenever a baseline was added or deleted, so every change that retired a baseline edited the same lines and two such changes conflicted. They are now included through `tests/phpstan/baselines/loader.php`, a PHP configuration that returns the baselines found in its directory, since NEON can only include files by name. A baseline takes effect when it is added and stops when it is deleted, with no edit of the configuration.

When regenerating, only the baselines being rewritten are suppressed: the generator passes their directory in the `WP_PHPSTAN_GENERATING_BASELINES` environment variable, and the loader there returns nothing. This works even when the configuration is included by another, such as a local `phpstan.neon`, which previously left every baseline in force and got them deleted. Baselines in other directories stay in force, so a stricter configuration with baselines of its own records only the errors the default ones do not cover.

Developed in #16049.
Follow-up to r63019, r63294, r65372.

Props westonruter, apermo.
See #65817.


git-svn-id: https://develop.svn.wordpress.org/trunk@65566 602fd350-edb4-49c9-b593-d223f7449a82
wporg-sync pushed a commit to WordPress/WordPress that referenced this pull request Nov 17, 2026
The per-identifier baselines were listed one by one in `phpstan.neon.dist`, between markers that `composer phpstan:baselines` rewrote whenever a baseline was added or deleted, so every change that retired a baseline edited the same lines and two such changes conflicted. They are now included through `tests/phpstan/baselines/loader.php`, a PHP configuration that returns the baselines found in its directory, since NEON can only include files by name. A baseline takes effect when it is added and stops when it is deleted, with no edit of the configuration.

When regenerating, only the baselines being rewritten are suppressed: the generator passes their directory in the `WP_PHPSTAN_GENERATING_BASELINES` environment variable, and the loader there returns nothing. This works even when the configuration is included by another, such as a local `phpstan.neon`, which previously left every baseline in force and got them deleted. Baselines in other directories stay in force, so a stricter configuration with baselines of its own records only the errors the default ones do not cover.

Developed in WordPress/wordpress-develop#16049.
Follow-up to r63019, r63294, r65372.

Props westonruter, apermo.
See #65817.

Built from https://develop.svn.wordpress.org/trunk@65566


git-svn-id: http://core.svn.wordpress.org/trunk@64639 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants