Skip to content

Project config: preserve resources and localized files - #4571

Merged
mathjazz merged 21 commits into
mozilla:mainfrom
ayshushus:2844-config
Oct 17, 2026
Merged

mathjazz merged 21 commits into
mozilla:mainfrom
ayshushus:2844-config

Conversation

@ayshushus

Copy link
Copy Markdown
Contributor

Fix #2844
Fix #4385

  • Current resources whose path is no longer in paths.ref_paths now go through mark_as_obsolete().
  • Localized files are now kept.
  • If the project uses a config and a config file is in the diff, the source file is now added to the diff along with the existing localized files.
  • Add a holistic test.

@ayshushus
ayshushus requested a review from mathjazz October 6, 2026 04:06

@mathjazz mathjazz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the patch! Left a few comments inline.

removed_paths = {
get_db_path(paths, join(checkout.path, co_path)) for co_path in checkout.removed
}
if isinstance(paths, L10nConfigPaths):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marking entities obsolete here can't be undone later. When a path comes back, update_resources() un-obsoletes the resource but not its entities, so it creates a second copy of every Entity, and the old translations, comments and reviews stay stuck on the hidden ones.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this effectively #4128? Based on your comment there, I'm applying a content (value & properties) match that is LIFO/newest copy. Also adding a "Restored" log.

Comment thread pontoon/sync/core/entities.py Outdated
get_db_path(paths, join(checkout.path, co_path)) for co_path in checkout.removed
}
if isinstance(paths, L10nConfigPaths):
ref_db_paths = {get_db_path(paths, ref_path) for ref_path in paths.ref_paths}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ref_paths is just whatever the glob matched this run, so a typo in reference (or a source folder moved one commit before the config is updated) makes it empty and wipes out the whole project in one sync.

Shall we refuse to obsolete when the list covers all or nearly all resources?

Comment thread pontoon/sync/core/paths.py Outdated
current = set(project.resources.current().values_list("path", flat=True))
for ref_path in paths.ref_paths:
if (
relpath(ref_path, paths.ref_root) in current

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This skips references we already track, so if only the l10n target moves (e.g. {locale}/a.ftl → l10n/{locale}/a.ftl), the files at the new location never get added to target.changed and their translations aren't picked up.

@eemeli eemeli left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A partial review, happy to have Matjaž take the lead here.

Comment thread pontoon/sync/core/paths.py
Comment thread pontoon/sync/core/entities.py Outdated
Comment thread pontoon/sync/core/entities.py Outdated
Comment thread pontoon/sync/core/entities.py Outdated

@mathjazz mathjazz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, nice work!

Comment thread pontoon/sync/core/checkout.py Outdated
Comment thread pontoon/sync/core/entities.py Outdated
Comment thread pontoon/sync/core/entities.py Outdated
Comment thread pontoon/sync/core/paths.py Outdated
Comment thread pontoon/sync/core/paths.py Outdated
Comment thread pontoon/sync/core/paths.py Outdated
Comment thread pontoon/sync/core/paths.py Outdated
Comment thread pontoon/sync/tests/test_e2e.py Outdated
Comment thread pontoon/sync/tests/test_e2e.py Outdated
Comment thread pontoon/sync/tests/test_entities.py Outdated
@ayshushus
ayshushus requested a review from eemeli October 14, 2026 04:42
Comment thread pontoon/sync/core/paths.py Outdated
Comment thread pontoon/sync/core/entities.py Outdated
Comment thread pontoon/sync/core/paths.py Outdated
Comment thread pontoon/sync/core/paths.py Outdated
@ayshushus
ayshushus requested a review from eemeli October 14, 2026 18:18
@mathjazz
mathjazz merged commit 0415540 into mozilla:main Oct 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Project config: strings in new file appeared only after forced sync Resources removed from project configuration are not removed from Pontoon

3 participants