chore: version packages - #17
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.
Releases
@moderation-api/unicode-spoofing@0.4.0
Minor Changes
#16
301b60cThanks @chrisdengso! - Separate decode damage from spoofing with a newencoding_damagesignal.U+FFFD used to count as
illegal, so a message whose name field had beenmangled upstream —
Hi Jos�� Luis— reported as a spoofing attack. Across a115-row sample of real SMS traffic that was 61% of all flags.
U+FFFD is a decoder's output, never an author's input: whatever the original
bytes were, they are gone by the time the character exists, so it can carry no
payload. It now reports as
encoding_damage, which is included insignalsand
wordsbut deliberately does not setspoofed, and the normalizerleaves it in place rather than silently repairing a corrupted message.
SpoofSignalgains a member andsignalsgains a key, so an exhaustiveswitchor a stricttoEqualon the signals object will need updating.Genuinely illegal code points (NUL, C1 controls, non-characters) are unchanged.
A
SPOOFING_SIGNALSconstant is exported alongsideSPOOF_SIGNALS— the samelist minus
encoding_damage— so callers can partition the two the same wayspoofeddoes.#16
301b60cThanks @chrisdengso! - Judge Latin words on UTS #39 Identifier_Status instead of skeleton luck.A word already written in Latin cannot be impersonating Latin, so the
cross-script skeleton test measured only whether its fold happened to reach
ASCII — which it does for ordinary European orthography, and arbitrarily.
æis a ligature that UTS #39 dissolves toae, whileøkeeps its stroke asa combining mark and never reaches ASCII, so
Ægirwas reported as a disguisedword and
Ålborgwas not. Same alphabet, opposite verdicts.A Latin word now needs at least one character Unicode marks Restricted in
IdentifierStatus.txtbefore its skeleton counts.Ægir,Þór,Straße,cœur,ısıtırandHawaiʻiare all Allowed and pass through;pɑypal(U+0251 IPA ALPHA) and
first(a compatibility ligature) are Restricted and arestill caught — intra-Latin homoglyphs that no script comparison can see.
The gate applies ONLY to Latin words. Cyrillic
а/оare Allowed too, beingordinary Russian, so applying it everywhere would let
раураlwalk through.Adds
src/data/identifier-status.generated.ts(~7 KB) andscripts/generate-identifier-status.mjs;npm run generate:datarefreshesboth tables.
#16
301b60cThanks @chrisdengso! - Stop reporting zero-width runs that are isolated in whitespace.A zero-width character evades a filter by splitting a word (
fr<ZWSP>ee).A run with whitespace (or a string boundary) on BOTH sides splits nothing —
the break it would create is already there — so it changes neither rendering
nor tokenization. Rich-text editors leave these in templates constantly, and
reporting them told senders their own newsletter was an attack.
Requiring isolation on both sides is the safety of the rule. One-sided contact
still does work:
admin<ZWSP>renders asadminbut compares unequal to it,and
<ZWSP>Valenciaglues to the front of a token the same way, so both arestill reported. Runs longer than the new exported
ZERO_WIDTH_INERT_RUN(4)are reported wherever they sit, because length alone makes a run a payload
channel regardless of position.
Scoped to genuinely zero-width, non-reordering characters (U+200B, U+200C,
U+200D, U+2060, U+FEFF). Bidi controls, tag characters and blank glyphs are
untouched — Trojan Source and ASCII smuggling detection are unaffected.