Skip to content

Author names: display_name vs sort_name, drop ambiguous first/last name - #105

Merged
benoit74 merged 1 commit into
mainfrom
benoit74/revisit-author-labels
Nov 2, 2026
Merged

benoit74 merged 1 commit into
mainfrom
benoit74/revisit-author-labels

Conversation

@benoit74

@benoit74 benoit74 commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The Gutenberg bug (#64) was a symptom of a deeper issue: first_name/last_name are ambiguous, source-specific concepts (not every source has them, and "last name" doesn't mean the same thing everywhere), and core's machinery had no business dealing with them. That's accidental complexity that isn't actually useful to the UI, and it's exactly what let the suffix-scrambling bug happen in the first place.

Core (and the UI) only ever need two things about an author's name:

  • display_name: a nicely formatted string to show readers, e.g. Henry Cranke Andrews (active 1799-1828).
  • sort_name: an opaque, technical key for unambiguous sorting (last name, then first name, then any disambiguating text a source provides), e.g. Andrews, Henry Cranke, active 1799-1828. Never displayed.

Changes:

  • Creator.name → display_name; Creator.sort_name is now that composite sort key instead of just a last name, so two authors sharing a last name (e.g. Alice Clarke / Bob Clarke) sort correctly instead of comparing equal.
  • Dropped Creator.first_name/last_name entirely. Gutenberg and OpenTextbooks still parse first/last locally just long enough to build display_name and sort_name, then discard them; nothing downstream needs the decomposed form.
  • Renamed the JSON/UI field the same way: AuthorPreview.name → display_name (displayName in JSON), matching the Python side, and updated every UI consumer.
  • Fixed two spots that were sorting authors by the display string instead of the sort key (same class of bug), caught while touching those call sites for the rename.
  • Gutenberg's RDF parser itself: stop reversing/scrambling the free-text segments after the first comma (e.g. "active 1799-1828") into the first name; keep them as a disambiguating suffix shown in parentheses.

Fixes #64

@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.23077% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.07%. Comparing base (d46f27a) to head (1289a2b).

Files with missing lines Patch % Lines
scraper/src/papers2zim/core/index_builder.py 83.33% 1 Missing ⚠️
.../src/papers2zim/sources/onemorelibrary/metadata.py 66.66% 0 Missing and 1 partial ⚠️
...r/src/papers2zim/sources/opentextbooks/metadata.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #105      +/-   ##
==========================================
+ Coverage   72.63%   73.07%   +0.43%     
==========================================
  Files          63       63              
  Lines        3972     3996      +24     
  Branches      626      633       +7     
==========================================
+ Hits         2885     2920      +35     
+ Misses        897      891       -6     
+ Partials      190      185       -5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@benoit74 benoit74 changed the title Gutenberg: stop scrambling author names with a free-text suffix Author names: display_name vs sort_name, drop ambiguous first/last name Oct 30, 2026
@benoit74 benoit74 self-assigned this Oct 30, 2026
@benoit74

Copy link
Copy Markdown
Contributor Author

@VikramAditya33 interested to have your feedback on this one ; I tried to simplify stuff, but I might have missed something.

@VikramAditya33 VikramAditya33 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @benoit74, I like the display_name/sort_name idea and couldn't find anything still using the removed fields. A few thoughts from trying it on the PG catalog are inline, none of them blockers. Let me know your thoughts as well.

Comment thread scraper/src/papers2zim/core/utils.py Resolved
Comment thread scraper/tests/test_rdf.py Resolved
Comment thread scraper/tests/test_rdf.py Resolved
PG author names with more than one comma (e.g. "Andrews, Henry Cranke,
active 1799-1828") had everything after the first comma reversed and
folded into the first name, producing garbled names like "active
1799-1828 Henry Cranke Andrews". The underlying bug was accidental
complexity: first_name/last_name are ambiguous, source-specific
concepts that core (and the UI) had no business dealing with, and
that's exactly what let the scrambling happen in the first place.

- Gutenberg's RDF parser now splits last name / first name / a
  free-text suffix properly (no more reversing), and renders a
  recognized title before the name ("Sir Thomas Browne") or a
  generational suffix after it ("Horatio Alger Jr.") instead of always
  parenthesizing; anything else (dates, "Saint", other titles, role
  annotations, ...) still defaults to parentheses. Verified against
  the real PG catalog (40,774 unique author strings).
- Core only deals with two name concepts now: `Creator.display_name`
  (the formatted string shown to readers, e.g. "Henry Cranke Andrews
  (active 1799-1828)") and `Creator.sort_name` (the last name alone,
  used only to sort, never displayed). Dropped the now-unused
  first_name/last_name fields and the JSON/UI's ambiguous `name` field
  is now `displayName`, matching the Python side.
- Fixed two UI spots that were sorting authors by the display string
  instead of the sort key (same class of bug the author list already
  had fixed): the home page's no-popularity author fallback, and the
  book list's "sort by author" option.
- Stopped mentioning the author twice in a book's full-text search
  index entry when it has no description.

Fixes #64
@benoit74
benoit74 force-pushed the benoit74/revisit-author-labels branch from 01891dd to 1289a2b Compare November 2, 2026 16:30
@benoit74
benoit74 merged commit 6a4c773 into main Nov 2, 2026
9 checks passed
@benoit74
benoit74 deleted the benoit74/revisit-author-labels branch November 2, 2026 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Revisit author labels

2 participants