Skip to content

A family-first name_order stops the leading particle run at the first name word - #398

Merged
derek73 merged 3 commits into
masterfrom
fix/395-order-aware-particle-stop
Aug 18, 2026
Merged

A family-first name_order stops the leading particle run at the first name word#398
derek73 merged 3 commits into
masterfrom
fix/395-order-aware-particle-stop

Conversation

@derek73

@derek73 derek73 commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Implements #395: a leading never-given particle run stops at the first name word when a family-first name_order is declared. Three commits — fixture, pin, change.

                          default (unchanged)      FAMILY_FIRST          FAMILY_FIRST_GIVEN_LAST
de Mesnil Juan            family='de Mesnil Juan'  family='de Mesnil'    family='de Mesnil'
                                                   given='Juan'          given='Juan'
de la Cruz Juan Carlos    family=whole             given='Juan'          middle='Juan'
                                                   middle='Carlos'       given='Carlos'

Where it lives

Neither of the two sites that failed before. PR #391 put the claim in assign and found assign cannot split an existing piece; PR #394 put it in grouping and took four regressions. The piece is the obstacle: de la Cruz Juan Carlos groups as [de][la Cruz Juan Carlos] once the ambiguous la chains forward, so any rule stopping "after the first name word" must cut inside a piece.

post_rules can. Roles are per token, and nothing downstream reads pieces — measured, the only readers are _assign (which runs before it) and post_rules itself. So grouping stays order-independent and the 2026-08-16 keystone survives everywhere except the reach of this one fold.

The order is read, not re-derived

assign now records the order it actually used on ParseState.order, and the fold keys on that. policy.name_order would have been wrong: a script_orders entry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already sits one function away, where the PARTICLE_OR_GIVEN emitter reads the role off the token rather than assuming given.

It counts units, not tokens

A conjunction join (P3) and a bound given-name pair (P5) each count once, so the stop cannot land inside one:

de la Vega y Santos Juan   ->  family='de la Vega y Santos'   given='Juan'
ibn Awf abdul Rahman       ->  family='ibn Awf'               given='abdul Rahman'
de la Cruz Juan y Eva      ->  family='de la Cruz'            given='Juan y Eva'

Both joins are read back off the tags, not the pieces — the prefix chain has already merged the joined piece into a longer one by the time the fold runs. This makes rules.md#P3's one-word clause executable, which it had no rule to demonstrate when it was written last week.

Verification, and what it is not

The differential harness reports 0 unexplained at every baseline — that is evidence for the accepted cost, not for the feature. The corpus runs under the default policy against 1.4.0, which has no name_order, so it structurally cannot see this change.

What it can see, measured against 2.1.0 with a worktree at the tag: 1 corpus name moves under each family-first order (de Mesnil Garcia → family de Mesnil, given Garcia), 0 in the default order, and the release log's "eight names under each family-first order" count is unchanged since that name was already in the eight.

The verification that counts is tests/v2/cases.py's two-leftover rows. Mutation-checked: replacing _name_positions(order, …) with a hardcoded layout that discards name_order fails exactly one row and nothing else in the suite — on master that mutant passes everything.

Commits

  1. fixture — the module's reduced _LEX misclassified three words against the shipped sets: la was never-given where it ships as ambiguous (in the module whose subject is how far a particle run chains), dr was absent, md was in the wrong suffix bucket. _parsed now compares its own tagging against Lexicon.default() on every input and fails with the divergence named. It caught two of my own new tests during this PR. Also Dr. de MD MesnilMr de MD Mesnil: a period makes any opening abbreviation a title by shape (H2), so the dotted title proved nothing about vocabulary.
  2. pin — the two-leftover shape, in a real Spanish listing form (Apellidos Nombres keeps the particle in place; Dutch would have been wrong, since Jong, Jan Pieter de moves it behind the given name — rule P6's shape, not this one). Committed before the change, recording that all three orders agreed beforehand.
  3. change — implementation, docs, release log.

Docs amended in the same diff

rules.md#P1 loses both deviates: #364 markers and gains six order-annotated examples; its "the reading holds under every declared name order" sentence is rewritten, since that is the sentence decisions.md#P1 superseded. rules.md#P2 gains the qualifier that the fold may take only part of the final group, plus an interacts: line it never had. P3 and P5 gain _post_rules.py as a second implementation site. decisions.md#P1 records where the stop lives and why, closing the question the supersession left open.

3502 tests pass, mypy clean.

Closes #395

🤖 Generated with Claude Code

derek73 and others added 2 commits August 18, 2026 00:50
A reduced lexicon is the convention in every pipeline stage module and
stays. What it must not do is classify a word DIFFERENTLY from the
shipped sets: the test then reads one name and parses another, and
passes for a reason its author never sees. Three words did that here,
found by the guard this commit adds, not by inspection:

  dr  absent, ships as a title AND a suffix word
  md  suffix_words here, ships as a title and a suffix ACRONYM
  la  never-given here, ships as AMBIGUOUS -- so "de la Vega" was
      chaining through the wrong kind of particle, in the module whose
      subject is how far a leading particle run chains

`_parsed` now compares its own vocabulary tagging against
Lexicon.default() for every input and fails with the divergence named.
Mutation-verified: restoring `la` to the never-given half fires it.

'Dr. de MD Mesnil' becomes 'Mr de MD Mesnil'. A period makes any
opening abbreviation a title by shape (rules.md#H2), so the dotted
title passed whether or not the fixture held the word -- the test could
not distinguish vocabulary from shape. Mutation-verified live: removing
the piece-skip from _leading_name_piece now fails it, giving
family='de', given='MD' exactly as the comment says.

Refs #395

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`de la Cruz Juan Carlos` is the only shape where FAMILY_FIRST and
FAMILY_FIRST_GIVEN_LAST can disagree with each other -- with one
leftover both send it to `given`. It appears in no case row, no unit
test and no rule example, which is why PR #394 could discard
name_order from the leftover placement and still pass the whole suite.

Spanish because the listing is real: "Apellidos Nombres" keeps the
particle in place. Dutch would have been wrong for this row -- its
family-first listing moves the particle behind the given name ("Jong,
Jan Pieter de"), which is rule P6's shape, not this one. The run also
reaches 'Cruz' THROUGH ambiguous 'la', so the row doubles as the chain
a stop keyed on never-given membership would break.

Pinned before the change because what it records is surprising: today
all three orders agree, each taking the whole name into the family.
The default-order row is the accepted cost of #395's direction and
must NOT move when it lands; 1.4.0 gives last 'de la Cruz Juan Carlos'
too, so it is parity.

The two family-first rows are core-only -- name_order is in
_UNTRANSLATED, having no v1 Constants spelling -- so they join
_CORE_ONLY_IDS, whose allowlist is what keeps that skip from being
silent.

Refs #395

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@derek73 derek73 self-assigned this Aug 18, 2026
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.53%. Comparing base (25180f5) to head (4627604).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #398      +/-   ##
==========================================
+ Coverage   98.51%   98.53%   +0.02%     
==========================================
  Files          44       44              
  Lines        2895     2942      +47     
==========================================
+ Hits         2852     2899      +47     
  Misses         43       43              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@derek73 derek73 added this to the v2.2 milestone Aug 18, 2026
@derek73
derek73 force-pushed the fix/395-order-aware-particle-stop branch 3 times, most recently from 5b47cc2 to 8ea126e Compare August 18, 2026 09:04
A never-given particle opening the name took every remaining word into
the family in all three orders. Under a declared family-first order it
now takes one name word and leaves the rest to the order:

    FAMILY_FIRST            "de Mesnil Juan" -> family 'de Mesnil',
                                                given  'Juan'
    FAMILY_FIRST_GIVEN_LAST "de la Cruz Juan Carlos" -> middle 'Juan',
                                                       given  'Carlos'

Declaring a family-first order asserts that what follows the family is
not more surname, which is the question the stopping point asks. The
DEFAULT order is unchanged and that is the accepted cost: with nothing
declared, "de Mesnil Juan" has the shape of "pennie von bergen
wessels", whose whole text is the surname. Callers who mean otherwise
write the comma, which already parses that way.

Where it lives, and why not the two sites that failed before: the fold
in post_rules. Grouping was PR #394 and assignment PR #391 -- and the
piece is the obstacle, since "de la Cruz Juan Carlos" groups as [de]
[la Cruz Juan Carlos] once the ambiguous particle chains, so the stop
must cut INSIDE a piece. post_rules can: roles are per token, and
nothing downstream reads pieces (measured -- only _assign, which runs
before it). Grouping stays order-independent.

The order is read, not re-derived. assign records the order it used on
ParseState.order and the fold keys on that; policy.name_order would
disagree with the roles assign already wrote whenever a script_orders
entry overrides it.

The run counts UNITS: a conjunction join (P3) and a bound given-name
pair (P5) each count once, so "de la Vega y Santos Juan" cannot stop
between Vega and Santos, and "abdul Rahman" cannot be halved. Both are
read off the tags -- the prefix chain has already merged the joined
piece away by then.

Measured: one differential corpus name moves, "de Mesnil Garcia",
under each family-first order; all 751 are byte-identical in the
default order. The corpus cannot see more than that -- it runs under
the default policy against 1.4.0, which has no name_order -- so the
verification that counts is the two-leftover case rows, which
mutation-checking confirms are the only thing in the suite that fails
when name_order is discarded from the leftover placement.

Closes #395

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@derek73
derek73 force-pushed the fix/395-order-aware-particle-stop branch from 8ea126e to 4627604 Compare August 18, 2026 09:55
@derek73
derek73 merged commit 9571011 into master Aug 18, 2026
11 checks passed
@derek73
derek73 deleted the fix/395-order-aware-particle-stop branch August 18, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A leading particle run should stop at the first name word when name_order is family-first

1 participant