A family-first name_order stops the leading particle run at the first name word - #398
Merged
Merged
Conversation
A reduced lexicon is the convention in every pipeline stage module and
stays. What it must not do is classify a word DIFFERENTLY from the
shipped sets: the test then reads one name and parses another, and
passes for a reason its author never sees. Three words did that here,
found by the guard this commit adds, not by inspection:
dr absent, ships as a title AND a suffix word
md suffix_words here, ships as a title and a suffix ACRONYM
la never-given here, ships as AMBIGUOUS -- so "de la Vega" was
chaining through the wrong kind of particle, in the module whose
subject is how far a leading particle run chains
`_parsed` now compares its own vocabulary tagging against
Lexicon.default() for every input and fails with the divergence named.
Mutation-verified: restoring `la` to the never-given half fires it.
'Dr. de MD Mesnil' becomes 'Mr de MD Mesnil'. A period makes any
opening abbreviation a title by shape (rules.md#H2), so the dotted
title passed whether or not the fixture held the word -- the test could
not distinguish vocabulary from shape. Mutation-verified live: removing
the piece-skip from _leading_name_piece now fails it, giving
family='de', given='MD' exactly as the comment says.
Refs #395
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`de la Cruz Juan Carlos` is the only shape where FAMILY_FIRST and FAMILY_FIRST_GIVEN_LAST can disagree with each other -- with one leftover both send it to `given`. It appears in no case row, no unit test and no rule example, which is why PR #394 could discard name_order from the leftover placement and still pass the whole suite. Spanish because the listing is real: "Apellidos Nombres" keeps the particle in place. Dutch would have been wrong for this row -- its family-first listing moves the particle behind the given name ("Jong, Jan Pieter de"), which is rule P6's shape, not this one. The run also reaches 'Cruz' THROUGH ambiguous 'la', so the row doubles as the chain a stop keyed on never-given membership would break. Pinned before the change because what it records is surprising: today all three orders agree, each taking the whole name into the family. The default-order row is the accepted cost of #395's direction and must NOT move when it lands; 1.4.0 gives last 'de la Cruz Juan Carlos' too, so it is parity. The two family-first rows are core-only -- name_order is in _UNTRANSLATED, having no v1 Constants spelling -- so they join _CORE_ONLY_IDS, whose allowlist is what keeps that skip from being silent. Refs #395 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #398 +/- ##
==========================================
+ Coverage 98.51% 98.53% +0.02%
==========================================
Files 44 44
Lines 2895 2942 +47
==========================================
+ Hits 2852 2899 +47
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
derek73
force-pushed
the
fix/395-order-aware-particle-stop
branch
3 times, most recently
from
August 18, 2026 09:04
5b47cc2 to
8ea126e
Compare
A never-given particle opening the name took every remaining word into
the family in all three orders. Under a declared family-first order it
now takes one name word and leaves the rest to the order:
FAMILY_FIRST "de Mesnil Juan" -> family 'de Mesnil',
given 'Juan'
FAMILY_FIRST_GIVEN_LAST "de la Cruz Juan Carlos" -> middle 'Juan',
given 'Carlos'
Declaring a family-first order asserts that what follows the family is
not more surname, which is the question the stopping point asks. The
DEFAULT order is unchanged and that is the accepted cost: with nothing
declared, "de Mesnil Juan" has the shape of "pennie von bergen
wessels", whose whole text is the surname. Callers who mean otherwise
write the comma, which already parses that way.
Where it lives, and why not the two sites that failed before: the fold
in post_rules. Grouping was PR #394 and assignment PR #391 -- and the
piece is the obstacle, since "de la Cruz Juan Carlos" groups as [de]
[la Cruz Juan Carlos] once the ambiguous particle chains, so the stop
must cut INSIDE a piece. post_rules can: roles are per token, and
nothing downstream reads pieces (measured -- only _assign, which runs
before it). Grouping stays order-independent.
The order is read, not re-derived. assign records the order it used on
ParseState.order and the fold keys on that; policy.name_order would
disagree with the roles assign already wrote whenever a script_orders
entry overrides it.
The run counts UNITS: a conjunction join (P3) and a bound given-name
pair (P5) each count once, so "de la Vega y Santos Juan" cannot stop
between Vega and Santos, and "abdul Rahman" cannot be halved. Both are
read off the tags -- the prefix chain has already merged the joined
piece away by then.
Measured: one differential corpus name moves, "de Mesnil Garcia",
under each family-first order; all 751 are byte-identical in the
default order. The corpus cannot see more than that -- it runs under
the default policy against 1.4.0, which has no name_order -- so the
verification that counts is the two-leftover case rows, which
mutation-checking confirms are the only thing in the suite that fails
when name_order is discarded from the leftover placement.
Closes #395
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
derek73
force-pushed
the
fix/395-order-aware-particle-stop
branch
from
August 18, 2026 09:55
8ea126e to
4627604
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements #395: a leading never-given particle run stops at the first name word when a family-first
name_orderis declared. Three commits — fixture, pin, change.Where it lives
Neither of the two sites that failed before. PR #391 put the claim in
assignand found assign cannot split an existing piece; PR #394 put it in grouping and took four regressions. The piece is the obstacle:de la Cruz Juan Carlosgroups as[de][la Cruz Juan Carlos]once the ambiguouslachains forward, so any rule stopping "after the first name word" must cut inside a piece.post_rulescan. Roles are per token, and nothing downstream readspieces— measured, the only readers are_assign(which runs before it) andpost_rulesitself. So grouping stays order-independent and the 2026-08-16 keystone survives everywhere except the reach of this one fold.The order is read, not re-derived
assignnow records the order it actually used onParseState.order, and the fold keys on that.policy.name_orderwould have been wrong: ascript_ordersentry can put the family first under a given-first policy, and the roles assign already wrote would then disagree with the roles the fold computes. The same reasoning already sits one function away, where thePARTICLE_OR_GIVENemitter reads the role off the token rather than assuminggiven.It counts units, not tokens
A conjunction join (P3) and a bound given-name pair (P5) each count once, so the stop cannot land inside one:
Both joins are read back off the tags, not the pieces — the prefix chain has already merged the joined piece into a longer one by the time the fold runs. This makes
rules.md#P3's one-word clause executable, which it had no rule to demonstrate when it was written last week.Verification, and what it is not
The differential harness reports 0 unexplained at every baseline — that is evidence for the accepted cost, not for the feature. The corpus runs under the default policy against 1.4.0, which has no
name_order, so it structurally cannot see this change.What it can see, measured against 2.1.0 with a worktree at the tag: 1 corpus name moves under each family-first order (
de Mesnil Garcia→ familyde Mesnil, givenGarcia), 0 in the default order, and the release log's "eight names under each family-first order" count is unchanged since that name was already in the eight.The verification that counts is
tests/v2/cases.py's two-leftover rows. Mutation-checked: replacing_name_positions(order, …)with a hardcoded layout that discardsname_orderfails exactly one row and nothing else in the suite — on master that mutant passes everything.Commits
_LEXmisclassified three words against the shipped sets:lawas never-given where it ships as ambiguous (in the module whose subject is how far a particle run chains),drwas absent,mdwas in the wrong suffix bucket._parsednow compares its own tagging againstLexicon.default()on every input and fails with the divergence named. It caught two of my own new tests during this PR. AlsoDr. de MD Mesnil→Mr de MD Mesnil: a period makes any opening abbreviation a title by shape (H2), so the dotted title proved nothing about vocabulary.Apellidos Nombreskeeps the particle in place; Dutch would have been wrong, sinceJong, Jan Pieter demoves it behind the given name — rule P6's shape, not this one). Committed before the change, recording that all three orders agreed beforehand.Docs amended in the same diff
rules.md#P1loses bothdeviates: #364markers and gains six order-annotated examples; its "the reading holds under every declared name order" sentence is rewritten, since that is the sentencedecisions.md#P1superseded.rules.md#P2gains the qualifier that the fold may take only part of the final group, plus aninteracts:line it never had.P3andP5gain_post_rules.pyas a second implementation site.decisions.md#P1records where the stop lives and why, closing the question the supersession left open.3502 tests pass, mypy clean.
Closes #395
🤖 Generated with Claude Code