perf(@angular/build): replace watchpack with @parcel/watcher and chokidar - #33656
perf(@angular/build): replace watchpack with @parcel/watcher and chokidar#33656clydin wants to merge 1 commit into
Conversation
a757036 to
9ed830d
Compare
7fc11f7 to
afa5b29
Compare
…idar This change replaces the watchpack file watching dependency in @angular/build with @parcel/watcher as the primary native file watcher, while falling back to chokidar (v4) for polling or unsupported environments. By leveraging @parcel/watcher's native C++ bindings (FSEvents, ReadDirectoryChangesW, inotify), file system watching is offloaded directly to OS kernel APIs, significantly reducing CPU and memory footprint during watch mode. Additionally, external directory watches are dynamically subsumed to minimize active native file handles, while early path filtering and event coalescing prevent redundant incremental rebuild triggers.
afa5b29 to
160848a
Compare
There was a problem hiding this comment.
Code Review
This pull request replaces watchpack with a custom file watcher implementation that utilizes @parcel/watcher with a chokidar fallback, updating the build action and adding unit tests. The review feedback identifies several critical issues in the new watcher implementation: path traversal bugs on Windows drive roots when using path.posix.dirname, incorrect case-sensitivity detection for non-existent directories, potential unhandled promise rejections from floating unsubscribe promises, and early termination of the cleanup loop if an unsubscription fails.
| /** | ||
| * Determines whether a file path lookup key or any of its parent directories are present in watchedFiles. | ||
| */ | ||
| function isPathWatched(fileLookupKey: string, watchedFiles: Set<string>): boolean { | ||
| if (watchedFiles.has(fileLookupKey)) { | ||
| return true; | ||
| } | ||
|
|
||
| let current = fileLookupKey; | ||
| while (true) { | ||
| const parent = path.posix.dirname(current); | ||
| if (parent === current) { | ||
| break; | ||
| } | ||
| if (watchedFiles.has(parent)) { | ||
| return true; | ||
| } | ||
| current = parent; | ||
| } | ||
|
|
||
| return false; | ||
| } |
There was a problem hiding this comment.
Using path.posix.dirname on Windows paths (even when normalized with forward slashes) introduces bugs when traversing up to a Windows drive root (e.g., c:/). Specifically, path.posix.dirname('c:/src') returns c:, and path.posix.dirname('c:') returns .. This means the traversal in isPathWatched will completely skip checking the drive root c:/ if it is present in watchedFiles.
We can solve this robustly by introducing a custom getDirectoryPath helper that correctly handles Windows drive roots and POSIX roots, and using it instead of path.posix.dirname.
/**
* Returns the parent directory of a normalized POSIX path, correctly handling Windows drive roots.
*/
function getDirectoryPath(posixPath: string): string {
const lastSlash = posixPath.lastIndexOf('/');
if (lastSlash === -1) {
return '.';
}
const dir = posixPath.slice(0, lastSlash);
if (dir === '' || dir.endsWith(':')) {
return dir + '/';
}
return dir;
}
/**
* Determines whether a file path lookup key or any of its parent directories are present in watchedFiles.
*/
function isPathWatched(fileLookupKey: string, watchedFiles: Set<string>): boolean {
if (watchedFiles.has(fileLookupKey)) {
return true;
}
let current = fileLookupKey;
while (true) {
const parent = getDirectoryPath(current);
if (parent === current) {
break;
}
if (watchedFiles.has(parent)) {
return true;
}
current = parent;
}
return false;
}| async ensureWatched(posixPath: string, lookupKey: string): Promise<void> { | ||
| if (isPathInside(lookupKey, this.rootDirLookupKey) || lookupKey === this.rootDirLookupKey) { | ||
| return; | ||
| } | ||
|
|
||
| const dirPath = path.posix.dirname(posixPath); | ||
| const dirKey = path.posix.dirname(lookupKey); | ||
| let dirEntry = this.externalDirFiles.get(dirKey); | ||
| if (!dirEntry) { | ||
| dirEntry = { dirPath, files: new Set<string>() }; | ||
| this.externalDirFiles.set(dirKey, dirEntry); | ||
| } | ||
| dirEntry.files.add(lookupKey); | ||
|
|
||
| await this.ensureDirWatched(dirPath, dirKey); | ||
| } | ||
|
|
||
| removeFile(lookupKey: string): void { | ||
| if (isPathInside(lookupKey, this.rootDirLookupKey) || lookupKey === this.rootDirLookupKey) { | ||
| return; | ||
| } | ||
|
|
||
| const dirKey = path.posix.dirname(lookupKey); | ||
| const dirEntry = this.externalDirFiles.get(dirKey); | ||
| if (dirEntry) { | ||
| dirEntry.files.delete(lookupKey); | ||
| if (dirEntry.files.size === 0) { | ||
| this.externalDirFiles.delete(dirKey); | ||
| const sub = this.extraSubscriptions.get(dirKey); | ||
| if (sub) { | ||
| this.extraSubscriptions.delete(dirKey); | ||
| void sub.unsubscribe(); | ||
|
|
||
| for (const [remainingDirKey, remainingDirEntry] of this.externalDirFiles.entries()) { | ||
| if (!this.isCoveredByExistingExternal(remainingDirKey)) { | ||
| void this.ensureDirWatched(remainingDirEntry.dirPath, remainingDirKey); | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Using path.posix.dirname on a file directly under the drive root (e.g., c:/file.txt) will return c:, which is treated as a relative path on Windows instead of the absolute drive root c:/. Replacing it with the custom getDirectoryPath helper ensures correct absolute path resolution.
Additionally, floating promises from sub.unsubscribe() and ensureDirWatched can lead to unhandled promise rejections if they fail. Adding .catch(() => {}) prevents potential crashes or test failures.
async ensureWatched(posixPath: string, lookupKey: string): Promise<void> {
if (isPathInside(lookupKey, this.rootDirLookupKey) || lookupKey === this.rootDirLookupKey) {
return;
}
const dirPath = getDirectoryPath(posixPath);
const dirKey = getDirectoryPath(lookupKey);
let dirEntry = this.externalDirFiles.get(dirKey);
if (!dirEntry) {
dirEntry = { dirPath, files: new Set<string>() };
this.externalDirFiles.set(dirKey, dirEntry);
}
dirEntry.files.add(lookupKey);
await this.ensureDirWatched(dirPath, dirKey);
}
removeFile(lookupKey: string): void {
if (isPathInside(lookupKey, this.rootDirLookupKey) || lookupKey === this.rootDirLookupKey) {
return;
}
const dirKey = getDirectoryPath(lookupKey);
const dirEntry = this.externalDirFiles.get(dirKey);
if (dirEntry) {
dirEntry.files.delete(lookupKey);
if (dirEntry.files.size === 0) {
this.externalDirFiles.delete(dirKey);
const sub = this.extraSubscriptions.get(dirKey);
if (sub) {
this.extraSubscriptions.delete(dirKey);
sub.unsubscribe().catch(() => {});
for (const [remainingDirKey, remainingDirEntry] of this.externalDirFiles.entries()) {
if (!this.isCoveredByExistingExternal(remainingDirKey)) {
this.ensureDirWatched(remainingDirEntry.dirPath, remainingDirEntry.dirPath).catch(() => {});
}
}
}
}
}
}| async close(): Promise<void> { | ||
| try { | ||
| if (this.pendingSubscriptions.size > 0) { | ||
| await Promise.allSettled(Array.from(this.pendingSubscriptions.values())); | ||
| } | ||
| for (const sub of this.extraSubscriptions.values()) { | ||
| await sub.unsubscribe(); | ||
| } | ||
| } finally { | ||
| this.extraSubscriptions.clear(); | ||
| this.pendingSubscriptions.clear(); | ||
| this.externalDirFiles.clear(); | ||
| } | ||
| } |
There was a problem hiding this comment.
If any of the sub.unsubscribe() calls throw an error during close(), the loop will terminate early, leaving the remaining subscriptions active. Using Promise.allSettled ensures that all subscriptions are unsubscribed regardless of individual failures.
async close(): Promise<void> {
try {
if (this.pendingSubscriptions.size > 0) {
await Promise.allSettled(Array.from(this.pendingSubscriptions.values()));
}
if (this.extraSubscriptions.size > 0) {
await Promise.allSettled(
Array.from(this.extraSubscriptions.values()).map((sub) => sub.unsubscribe()),
);
}
} finally {
this.extraSubscriptions.clear();
this.pendingSubscriptions.clear();
this.externalDirFiles.clear();
}
}| // Subsume any nested child subscriptions that are now covered by this parent subscription | ||
| for (const [childDir, childSub] of this.extraSubscriptions.entries()) { | ||
| if (childDir !== dirKey && isPathInside(childDir, dirKey)) { | ||
| this.extraSubscriptions.delete(childDir); | ||
| void childSub.unsubscribe(); | ||
| } | ||
| } | ||
| } else { | ||
| void sub.unsubscribe(); | ||
| } |
There was a problem hiding this comment.
Floating promises from childSub.unsubscribe() and sub.unsubscribe() can lead to unhandled promise rejections if they fail. Adding .catch(() => {}) ensures robustness.
| // Subsume any nested child subscriptions that are now covered by this parent subscription | |
| for (const [childDir, childSub] of this.extraSubscriptions.entries()) { | |
| if (childDir !== dirKey && isPathInside(childDir, dirKey)) { | |
| this.extraSubscriptions.delete(childDir); | |
| void childSub.unsubscribe(); | |
| } | |
| } | |
| } else { | |
| void sub.unsubscribe(); | |
| } | |
| // Subsume any nested child subscriptions that are now covered by this parent subscription | |
| for (const [childDir, childSub] of this.extraSubscriptions.entries()) { | |
| if (childDir !== dirKey && isPathInside(childDir, dirKey)) { | |
| this.extraSubscriptions.delete(childDir); | |
| childSub.unsubscribe().catch(() => {}); | |
| } | |
| } | |
| } else { | |
| sub.unsubscribe().catch(() => {}); | |
| } |
This change replaces the watchpack file watching dependency in @angular/build with @parcel/watcher as the primary native file watcher, while falling back to chokidar (v4) for polling or unsupported environments.
By leveraging @parcel/watcher's native C++ bindings (FSEvents, ReadDirectoryChangesW, inotify), file system watching is offloaded directly to OS kernel APIs, significantly reducing CPU and memory footprint during watch mode. Additionally, external directory watches are dynamically subsumed to minimize active native file handles, while early path filtering and event coalescing prevent redundant incremental rebuild triggers.