Skip to content

feat(scan): match author folders differing only by post-nominals - #990

Open
dny238 wants to merge 1 commit into
Listenarrs:canaryfrom
dny238:fix/author-postnominals
Open

dny238 wants to merge 1 commit into
Listenarrs:canaryfrom
dny238:fix/author-postnominals

Conversation

@dny238

@dny238 dny238 commented Sep 16, 2026

Copy link
Copy Markdown

Summary

Follow-up to #784. The book-boundary scan matcher compares a normalized author directory name against the audiobook's expected author tokens. When a professional or generational post-nominal appears on only one side — metadata Jane Doe, PhD vs a Jane Doe folder, or a Martin King Jr folder vs Martin King metadata — the author context previously failed to match.

Change

  • BuildExpectedAuthorTokens now also emits a post-nominal-stripped variant of each author.
  • HasAuthorContext strips post-nominals from the directory token before comparing.

Together these make the match symmetric in either direction. The change is additive — no previously-produced token is removed, so nothing that matched before stops matching.

Deliberately narrow (per @m4bard's over-matching concern on #784)

  • Post-nominals only. Initials are not expanded — L. M. stays distinct from Lucy Maud, avoiding the genuine widening that initials matching would introduce.
  • The suffix set excludes short degree abbreviations that collide with real name words (e.g. ma as in Yo-Yo Ma, do, ba, bs), so a surname is never silently dropped.
  • Stripping always leaves at least one word.

Suffix set: phd md dds dvm dpt edd psyd jd esq mba cpa llm llb mfa jr sr ii iii iv.

Scope boundary — what this intentionally does not cover

This is deliberately the narrow, trailing-post-nominal shape, not a general credential/name normalizer. Out of scope by design:

  • Leading-position credentials — e.g. a Dr. Jane Doe or M.Ed. Karla McLaren folder against a clean record. Stripping only walks backward from the end and stops at the first non-post-nominal word, so a credential in front of the name is never reached.
  • Initials / middle-initial collapse — e.g. John M. Gottman vs John Gottman, or L. M. vs Lucy Maud. These are a genuine widening (any author whose initials collide gets accepted) and stay out for the same reason initials matching does.

So the two worked examples in #784 are not closed by this PR — they need leading-position and initial handling, which carries the widening risk we agreed to avoid. Growing the list and reaching the leading position would be a separate change with its own argument about where stripping stops.

Known, accepted trade-off: Jr/Sr both strip, so Martin King Jr and Martin King Sr reduce to the same token. Exposure is negligible — author context only corroborates after a full title-segment match.

Tests (in the ScanFileDiscoveryTests harness)

  • Post-nominal in metadata matches a clean folder
  • A generational-suffix folder matches clean metadata
  • Ambiguous ma surname is not stripped (no false match)
  • Initials are not expanded (no false match)

All ScanFileDiscoveryTests green; architecture gates (file-focus, test conventions) green.

Coordination

@m4bard validated this against his public generator (both directions link, no over-reach, and the #784-v1 BasePath-collapse failure did not return). Thanks for running it.

🤖 Generated with Claude Code

Follow-up to Listenarrs#784. The book-boundary matcher compares a normalized author
directory name against expected author tokens; a professional or generational
post-nominal on either side (metadata "Jane Doe, PhD" vs a "Jane Doe" folder,
or a "Martin King Jr" folder vs "Martin King" metadata) previously blocked the
match.

BuildExpectedAuthorTokens now also emits a post-nominal-stripped variant, and
HasAuthorContext strips post-nominals from the directory token, making the
match symmetric. Changes are additive (no existing token is removed).

Scope is deliberately narrow, per @m4bard's over-matching concern on Listenarrs#784:
- Post-nominals only. Initials are NOT expanded ("L. M." stays distinct from
  "Lucy Maud"), which would over-match otherwise-distinct authors.
- The suffix set excludes short degree abbreviations that collide with real
  name words (e.g. "ma" as in Yo-Yo Ma), so surnames are never dropped.
- Stripping always leaves at least one word.

Tests: post-nominal in metadata matches a clean folder; a generational-suffix
folder matches clean metadata; an ambiguous "ma" surname is not stripped;
initials are not expanded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dny238
dny238 requested a review from a team September 16, 2026 05:35
@m4bard

m4bard commented Sep 17, 2026

Copy link
Copy Markdown

I ran it rather than leave you waiting on the offer.

Built your branch at 8f879cc and built Listenarrs/Listenarr@canary the same way for the base column. That is a630572e, which is your parent commit, so the only variable is your diff. Library is two public-domain books, both Arthur Conan Doyle, laid out {Author}/{Title}. One book is added, its BasePath is cleared so the scan walks the library root, then every linked file is mapped back to its true owner from the generator's manifest. The second book sits beside it in ordinary form with no record, so a correct scan links one file and nothing else.

Folder side: the record is clean, the author folder carries the credential

author folder on disk base a630572 8f879cc
Arthur Conan Doyle (no variant) 1 linked, own, 0 foreign 1 linked, own, 0 foreign
Arthur Conan Doyle, PhD 0 linked 1 linked, own, 0 foreign
Arthur Conan Doyle Jr 0 linked 1 linked, own, 0 foreign
Arthur Conan Doyle, CFP 0 linked 0 linked
Dr. Arthur Conan Doyle 0 linked 0 linked

Record side: the folder is clean, the record carries the credential

record author base a630572 8f879cc
Arthur Conan Doyle, PhD 0 linked 1 linked, own, 0 foreign
Arthur Conan Doyle, CFP 0 linked 0 linked

Both directions work, which is the symmetry your description claims, and I could not get it to over-reach: not one run on either image claimed a file belonging to the sibling. On the two linking folder rows BasePath lands on the variant author folder over the book's own title folder, /audiobooks/Arthur Conan Doyle, PhD/The Hound of the Baskervilles, rather than collapsing to a common parent. That was the failure mode on the first revision of #784 and it is not back.

Three cells are zero on your branch, so worth saying why they are not just a broken harness. The no-variant row links on the same image with the same command, so the apparatus reaches the author folder. And the CFP row differs from the PhD row by three characters in one folder name, on the same image, where PhD links. So the nulls are about the credential rather than about the run.

Where the tolerance stops

The curated list is the boundary, and #784's own examples sit outside it. The list in that report is PhD, MD, M.Ed., CFP and Jr. Three of those are in the set at ScanFileDiscovery.Metadata.cs:283-288 and two are not, which is the CFP row above. And StripTrailingPostNominals at :316-333 walks backwards from the end and stops at the first word that is not a post-nominal, so a credential in front of the name is never reached. That is the Dr. row, and it is also the shape of the first example in #784: Karla McLaren on disk for an M.Ed. Karla McLaren record. The second example, John M. Gottman against John Gottman PhD, needs the middle initial dropped rather than the credential, so it stays out by the same design decision that keeps initials out, which I think is the right call.

I am not arguing for a wider list. A curated list is defensible and the alternative rule, "strip any short trailing token", is exactly the kind of widening that caused the misattributions I measured on the first revision. But there are two reasonable shapes here and they read differently to a reviewer: the PR as it stands, which covers the common trailing credentials and leaves both of #784's worked examples unmatched, or the same mechanism with the list grown and the leading position handled, which closes #784's examples and needs its own argument about where stripping stops. Which one you want this PR to be is your call, and I would rather ask than assume, because either answer changes what the description should say.

There is also an asymmetry I read but did not run. MetadataMatchesAudiobook compares the raw normalized tag artist against the expected author tokens at :91-93. Since BuildExpectedAuthorTokens now emits the stripped record variant, a clean tag against a credentialled record matches there too, which is a free win. The reverse does not: a file tagged Jane Doe, PhD against a clean Jane Doe record has nothing stripping the tag side, because the folder-side fix at :222-226 lives in HasAuthorContext and only ever sees directory names. That is the tag fallback for candidates path discovery did not attribute, AudiobookScanService.Metadata.cs:69-72. I have not exercised it, so treat it as a code reading. If you want it covered I can add a tag state that writes a post-nominal into the artist tag and measure it the same way.

One more from reading rather than running, and I want to be careful with it because I could not measure it: jr and sr are both in the set and stripping runs until it hits a non-post-nominal word, so Martin King Jr and Martin King Sr reduce to the same token. A public-domain corpus cannot hold two authors of the same name sharing a book title, so I have no runtime evidence either way. The exposure looks small in any case, since author context is corroboration after the title segment has already matched in full.

Running it yourself

The variants are on the public generator now, so none of this depends on me:

git clone https://github.com/m4bard/listenarr-testdata && cd listenarr-testdata
python3 -m venv .venv && .venv/bin/pip install -e .

./tools/vet-against.sh --repo https://github.com/dny238/Listenarr.git \
    --branch fix/author-postnominals --tool attribution \
    --asin B0036HXZCO --layout author-title \
    --only-asin B0036HXZCO,B0036I51QQ \
    --folder-variant author-postnominal:B0036HXZCO

Swap the variant key for author-generational, author-postnominal-uncommon or author-honorific for the other folder rows, and generate_library.py --list-folder-variants prints the whole set. The record side is the same runner with --record-author-suffix ", PhD" and no variant; it reads the stored record back and refuses to continue if the suffix did not survive the add, since a metadata lookup quietly replacing the authors would otherwise look exactly like a clean result.

Disclosure: drafted with Claude Code at my direction; I read the cited code at the stated commit and reviewed this before posting.

@dny238

dny238 commented Sep 18, 2026

Copy link
Copy Markdown
Author

Thank you for building it rather than waiting on me — the two-image table with the sibling-attribution column is exactly the confirmation the diff can't give, and the BasePath row (variant author folder over the book's own title folder) is the specific thing I most wanted to see not regress.

On the scope question: A — I want this PR to stay the trailing-post-nominal shape, and I've rewritten the description to say so in as many words. Both of #784's worked examples stay out by design: M.Ed. Karla McLaren is a leading credential, which the backward walk never reaches, and John M. Gottman needs the middle initial dropped, which is the initials widening we agreed to keep out. Growing the list and reaching the leading position is a real change with its own "where does stripping stop" argument, and folding it in here would get this judged by the riskier half — the same trap as #784's first revision. If those examples are worth closing they deserve their own PR and their own evidence.

The two reads you flagged from code rather than runtime:

  • jr/sr collapsing Martin King Jr and Martin King Sr to one token — agreed, and I'm leaving it. The exposure is what you said: author context only corroborates after the title segment has already matched in full, so it can't promote a wrong book on its own. I've noted it as an accepted trade-off in the description rather than pretend it isn't there.
  • The MetadataMatchesAudiobook asymmetry — good catch, and I agree with your reading. The clean-tag-against-credentialled-record direction is a free win because BuildExpectedAuthorTokens feeds it; the reverse isn't stripped because this PR's folder-side change lives in HasAuthorContext and only sees directory names. I'd rather not reach into the tag-fallback path (AudiobookScanService.Metadata.cs:69-72) from this PR — it's a different surface and it would blur the boundary this one is trying to keep sharp. If it turns out to bite in practice it's a clean follow-up, and I'd take you up on the tag-state offer then rather than now.

Disclosure: drafted with Claude Code at my direction; I made the scope call and reviewed this before posting.

@m4bard

m4bard commented Sep 18, 2026

Copy link
Copy Markdown

Scope A works for me, and keeping the leading-credential case for its own PR with its own evidence is the right call. The tag-state offer still stands whenever you want it, no pushing it from my side. Worth saying: A is the narrower option and the harder one to show off, and you've picked it twice on this thread.

Disclosure: drafted with Claude Code at my direction; reviewed this before posting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants