Adopt @microbit/i18n-tools for translation management - #1308
Merged
Conversation
Replaces bin/compile-lang.mjs, crowdin-convert.cjs and tidy-lang.cjs with an i18n.config.mjs driving the shared tool. The Crowdin file is assumed to move to react-intl JSON like the other apps', so the Chrome JSON conversion step goes; the first upload replaces it in that format. update-translations.sh remains the one-shot orchestration across the stubs and pyright repositories, now calling the same tool in each rather than copying out of a project zip. The compiled src/messages/*.json are generated output: gitignored and compiled by prepare and before start, typecheck and build, as styled-system is. The translated lang files hold only what Crowdin has translated, so a change to an English string touches one file. CI checks the catalogs are tidy. Four translations rendered literal markup or were the wrong sentence entirely (three Catalan strings escaping <link> with "d'<link>", and the Korean webusb-error-reconnect-microbit-description); they are dropped so English shows until fixed in Crowdin, which is what the new download does with such strings. The lockfile still needs regenerating once @microbit/i18n-tools is published; the package.json change drops @formatjs/cli in its favour.
The weekly download opens or refreshes a pull request for the configured languages. Upload is run by hand from the Actions tab so a batch of copy can be translated together, or a single string left in English for now. Both need the org Crowdin token granted to this repo, and the download needs the repository setting allowing Actions to create pull requests.
The fork's own dependencies stay untouched, so update-translations.sh fetches @microbit/i18n-tools with npx there.
lang/ui.zh-cn.json becomes ui.zh-CN.json and so on, matching the ids the app passes around and Crowdin's own, and the compiled catalogs follow. The provider's per-language switch becomes a lookup in supportedLanguages and a template import, which Vite still splits per catalog, so adding a language no longer needs a code change here. The language server side keeps its lowercase convention: pyright.ts still lowercases for the typeshed.<lang>.json imports and the server's locale, matching the pyright fork and the stubs repository.
typeshed.zh-cn.json becomes typeshed.zh-CN.json, as the stubs repository now generates them. The language-server client takes the canonical id, resolved from the stored or linked value, and lowercases only the locale it sends to our pyright fork, whose message files stay lowercase. The API docs it returns are now labelled with the canonical id too, which the search indexing compares against the settings language; for languages with a region, such as zh-CN, that comparison could never have matched the lowercase label before.
All eight repos share one Crowdin token; starting them ten minutes apart keeps them off it at the same moment.
Pushes and pull requests made with GITHUB_TOKEN never start workflows, so the weekly sync pull request got no CI. A token from the microbit-i18n GitHub App, narrowed to contents and pull requests, does; its setup is in the @microbit/i18n-tools README. Files are staged before the change check so a newly added language's file counts as a change.
microbit-i18n download exits 2 when some files failed and the rest were written. The workflow carries on to the pull request on 0 or 2, then fails the job on 2 so the failure is still seen; anything else stops it before the pull request step.
microbit-i18n download --summary writes a Markdown account of the run: files written, failed downloads, and each translation left out for a placeholder problem with the English and translated text. The workflow appends it to the step summary, uses it as the pull request body (refreshed on each run), and adds a warning annotation when anything was left out.
The pull request had no reviewer, so it waited for someone to notice it.
GitHub rejected the file: the runner context is only available to steps.
The token, status handling, summary, commit, push and pull request steps now live in microbit-foundation/translations-sync-action; this workflow keeps only the install and the download command.
# Conflicts: # package-lock.json
The Crowdin file predates Crowdin's react-intl support and its strings carry screenshots and other hand-added context that a change of format would lose, so rather than replacing it in react-intl JSON the catalog declares `crowdinFormat: "chrome"` and the tool converts at upload and download, as bin/crowdin-convert.cjs used to. The local lang files were always react-intl and are unchanged. Drops the gitignore entry for the old script's crowdin/ output.
Collaborator
Author
|
Although I've merged this to help test workflow_dispatch, the update-translations.sh script will not work until we deal with similar PRs for stubs and pyright, and it's not really clear whether it's the right approach (vs say, shipping translations with releases of some sort and pulling in translations that way). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We've extended the tooling to support the weird use of Chrome JSON in this project (but with ICU messages) rather than change it in Crowdin, because my testing suggested we'd lose some translations even adding TM pre-translate to the mix, and more importantly we lose all the screenshots that Emma has added over the years to enrich context for translators.
Even though Crowdin JSON is supposed to use a placeholders field it seems like ICU messages are still recognised and provided with editing support in Crowdin, I guess because the format is very generic with just message/description so might be used in other scenarios.