Store result cache paths relative to the install location behind a toggle - #6190
Store result cache paths relative to the install location behind a toggle#6190SanderMuller wants to merge 4 commits into
Conversation
…ggle The result cache stores absolute paths in its meta, keys and stored objects, and compares metadata with a strict whole-array match, so a changed absolute prefix (a fresh CI checkout dir, a git worktree) throws the whole cache away even when the relative layout is identical. Add a bleeding-edge featureToggle, relativePathResultCache, that stores the paths relative to the phpstan install (%rootDir%) and re-absolutizes them against the current install on load. Only paths reachable from the anchor become relative; the rest stay absolute, following ccache's rule. Error gains relativizePaths()/absolutizePaths(), building on its existing immutable changeFilePath() pattern, and a new ResultCachePathTransformer handles the rest of the cache structure at the save/restore boundary. The toggle state is folded into the cache meta and CACHE_VERSION is bumped so flipping it or upgrading migrates with one cold run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| echo "$OUTPUT" | ||
| ../bashunit -a contains 'Composer metadata changed but no package versions changed; keeping the result cache.' "$OUTPUT" | ||
| ../bashunit -a contains 'Result cache restored. 1 file will be reanalysed.' "$OUTPUT" | ||
| - script: | |
There was a problem hiding this comment.
We need a separate e2e test showing result cache can be re-used after a git worktree is created
There was a problem hiding this comment.
Added in e5a2d61. It warms the cache in one checkout, creates a git worktree at a different absolute path with its own vendor (so %rootDir% points at the worktree), carries the warm cache in, and asserts the worktree run reuses it with 0 files reanalysed. So it proves the stored paths re-absolutize against the worktree's location.
One note on scope: the test copies the warm cache into the worktree rather than having PHPStan discover the main checkout's cache. Auto-discovery (reaching into the main checkout for the warm cache) is the separate, harder piece @ondrejmirtes flagged and it is not in this PR. This test covers what the PR actually implements, that a cache made available in the worktree is portable to it.
Warms the cache in one checkout, creates a git worktree at a different absolute path with its own phpstan install, carries the warm cache over, and asserts it is reused with 0 files reanalysed. Proves the relative paths re-absolutize against the worktree, the scenario the toggle targets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PHPStan's -vv progress, including the "Result cache restored" line, is written to stderr. The assertion captured stdout only, so it missed the message and failed even though the cache was reused. Redirect stderr into the captured output. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
I don't think this has to be behind bleeding edge. It's not a BC break, it should work the same way as before. It's a good point that some people might be doing something to the structure of the current result cache format to achieve similar things this PR aims to achieve, but that's out of BC surface anyway, and the end result should be that they should have to be able delete their tooling altogether, if we do a good job. As a bonus, more E2E result cache tests will test the new path. |
Per review: this is not a BC break. For a project analysed on the same machine the relativized paths re-absolutize to the exact same absolute paths, so behaviour is unchanged; the only difference is that a moved project (a CI checkout dir, a git worktree) now reuses the cache instead of discarding it. Drop the featureToggle and relativize/absolutize unconditionally. The CACHE_VERSION bump migrates old caches with one cold run, and every result cache e2e now exercises the new path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Done, made it the default and dropped the featureToggle. Since the on-disk format changes I bumped |
A first cut at portable result cache, to iterate on per the discussion. The result cache is currently non-portable because it stores absolute paths everywhere: meta embeds them, storage is keyed by them, and the metadata comparison is a strict whole-array match. So a fresh CI checkout dir or a git worktree, where the relative layout is identical but the absolute prefix changed, throws the whole cache away.
This makes the cache store paths relative to an anchor and re-absolutize them on load.
Approach
%rootDir%, which is the phar's directory, or the source checkout when running frombin/phpstan). This is the anchor @ondrejmirtes landed on. Because a Composer-installed phar sits at a fixed offset inside the project (vendor/phpstan/phpstan), the path from the anchor to the analysed code is stable across checkouts and worktrees, which is exactly what has to survive a move.ErrorgetsrelativizePaths()/absolutizePaths(), building on its existing immutablechangeFilePath()pattern. A newResultCachePathTransformerorchestrates the rest of the cache structure (meta, dependencies, the compoundlinesToIgnorekeys, projectConfig) using the existingParentDirectoryRelativePathHelperto relativize andFileHelper::absolutizePath()to invert.CACHE_VERSIONis bumped so old caches migrate with one cold run, and every result cache e2e now exercises the new path.The transform lives entirely at the cache I/O boundary:
save()relativizes just before writing,restore()absolutizes right afterrequire. The rest of the manager keeps working in absolute paths.Proof
ResultCachePathTransformerTest): relativize a representative cache payload against anchor A, absolutize against a different anchor B, and every path (error keys and objects, trait paths, dependency lists, the compound trait-context keys, meta, projectConfig) comes out rooted at B. Same-anchor round-trip is identity, and outside-anchor paths stay absolute.result-cache-relative-path): a cold run stores the analysed file relative to the anchor (asserted present, and asserted the absolute checkout path is absent), and a warm run re-absolutizes and reuses the cache with 0 files reanalysed.0 files will be reanalysed; with it off the same move reportsmetadata do not match: projectConfig, analysedPaths, executedFilesHashesand re-analyses everything. The committed e2e cannot relocate the anchor in the source harness, so that move is covered by the unit test rather than the fixture.Open question: the anchor
%rootDir%is portable when the phar moves with the project (Composer-installed phar, worktree, CI checkout). A globally-installedphpstan.pharoutside the project tree does not share a moving prefix with the code, so its relative offsets would not survive a move.%currentWorkingDirectory%moves with the project in those same scenarios and matches the issue's literal wording, at the cost of not covering PHPStan's own stub and config paths. The anchor is isolated in one method (getPathTransformer()plus the injected%rootDir%) so it is a one-line change to try cwd instead. This is the main thing worth settling.Scope of this first cut
Covered: the full storage surface (errors, locallyIgnoredErrors, linesToIgnore/unmatchedLineIgnores including compound trait keys, collectedData, dependencies, packageDependencies, exportedNodes, projectExtensionFiles) and the meta paths (analysedPaths, scannedFiles, composerLocks, composerInstalled including nested
install_path, executedFilesHashes, stubFiles, and projectConfigpaths/tmpDir).Not yet portable, and documented as follow-ups rather than silently missed:
excludePaths(stored asOptionalPathobjects), so a project relying on optional excludePaths still invalidates on a move.ResultCacheMetaExtensionhashes and collector payloads that embed absolute paths, which a core transform cannot see into. The Symfony container extension already foldskernel.project_dirinto its meta hash, so it needs its own portability handling.normalizePath()is lexical and does not resolve symlinks, so this needs thought before the toggle becomes the default.Closes phpstan/phpstan#8599