A tussenvoegsel after a family comma joins the family name

docs/design/decisions.md

@@ -80,6 +80,11 @@ Open: [#360](https://github.com/derek73/python-nameparser/issues/360) which part
 
 ### P6 — the trailing orphan particle
 
+- 2026-08-18 #379/#380 IMPLEMENTED — the rule shipped as stated, both `deviates:` markers out. Keyed on the token's VOCABULARY rather than its assigned role, which is what gives the S2 precedence its effect: assign reads a trailing `vd`/`mc`/`do` as a post-nominal, and the attachment overrides that after a comma. The words-to-spare guard is a PIECE test, not a count — every trailing piece that is wholly particles attaches, and the run must leave a GIVEN word ahead of it, so "Nguyen, Van" keeps its only given word. Not "a name word": see the defect below, where that wider reading passed on family text P1 had just produced. MIDDLE was in the test briefly and came out — 0 hits over 740,552 instrumented guard sites, and structurally unreachable, since the only rule that leaves a MIDDLE with no GIVEN ahead of it is gated on state.order, which assign never records on the family-comma path P6 runs on.
+  Measured: ONE of 751 corpus names moves ("Vega, Juan de la"), at all three baselines, and it needed a new ledger rule in each. The LEDGER rule's regex — not the parse rule, which has no regex and fires on the whole particle vocabulary through the tag — is a PARTIAL copy of PARTICLES, deliberately: the words that actually end a Dutch, Iberian or German listing, not all 70. The ledger guards forced that narrowing: the first draft used the neighbours' bare comma regex, which claimed 215 names and reached two protected exclusions. A member `des` had to come out as well, and for a reason worth keeping straight: it is not in PARTICLES at all, so it could copy nothing, and the text it reached was the opening of "Designated Judge David A. Ezra" rather than a person's name. `_reaches_non_vocabulary("des", PARTICLES)` prints the FRAGMENT it matched, `['Des']`, not the name containing it — which is how the retracted wording arose in the first place.
+  Two defects found in review, both from the rule's neighbours rather than from its own logic. P1's fold runs EARLIER in the same function and retags all of segment 1 to FAMILY, so a words-to-spare test written as "some name role remains" passed on family text P1 had just produced — and the attachment then hoisted the particle in front of a base it never preceded ("Smith, de Mesnil van" → 'van Smith de Mesnil'). 1.4 gives 'de Mesnil van Smith' for that input and 2.1 gives 'Smith de Mesnil van', so neither the defect nor the fix restores 1.4 — what the fix restores is 2.1, the pre-comma-family reading having moved back in 2.0. The test has to name GIVEN, which is what the rule says. And a post-nominal sits BEHIND the tussenvoegsel in this listing, so the backward walk stopped on it and the attachment never fired: "Berg, Jan van Jr." and "Berg, Jan van, Jr." parsed two ways on nothing but a comma. The walk now steps over a trailing piece holding no name, unless it is itself particle vocabulary — `vd` arrives suffix-roled and IS the run.
+  Fallout worth its own record: "Vega, Juan de la" was the input tests/test_initials.py used to build an ALL-PARTICLE middle name, and this rule takes that shape away from THAT input. It does not remove the shape — "Vega, Santa de la", "Berg, Van der" and "Nguyen, Van Le" all still produce one in the default order, because the trailing run swallows the given word too and the attachment then declines. The test is repointed at the last of those rather than deleted. Whether such a middle should exist at all is #402 — for a never-given particle it is #365's defect, and for an ambiguous one the parser reports it as a middle name and then declines to initial it, which is a miscount either way.
+
 - 2026-08-16 (order-precedence keystone; #379, #380, #365) — a particle ending the name has nothing to link forward to, and no particle is a name by itself, so it attaches to the family name standing beside it and renders BEFORE it. The distinction from a chain is what makes this a rule rather than an exception: a chained group has a head word and can be positioned; an orphan has no head, so position has nothing to work with.
 - Scope: the COMMA form only, deliberately. "Jong, Anke de" is unambiguous — the comma has already named the family. Without the comma the written shape is not settled: "Jong Anke de" may be a misformatted listing (arguably a missing comma under a declared family-first order) and "Jong de" may be a given name beside a particle. Those keep their positional reading and are not tracked as deviations.
 - The words-to-spare guard is load-bearing, not incidental. #379's own subject is "van", which is in the AMBIGUOUS half — so a rule keyed to never-given particles alone would not fix the issue it was filed for, while a rule with no guard breaks Vietnamese

docs/design/rules.md

@@ -245,24 +245,54 @@ P6. Rationale: a particle ending the name has nothing to link
     forward to, so it is not doing a particle's work there. A
     never-given particle in that position cannot be a name at all
     and must belong to the family written beside it; an ambiguous
-    particle could genuinely be the name (Vietnamese "Van"), which
-    is what the words-to-spare test below is for, not an
-    afterthought to it. Dutch and Flemish names are listed exactly
+    particle could genuinely be the name (Vietnamese "Van"), and
+    after a comma there is no signal that separates the two
+    readings. Dutch and Flemish names are listed exactly
     this way ("Beethoven, Ludwig van"), the tussenvoegsel trailing
     the given name but belonging to the surname.
     Where a family comma has already named the family, a particle
     ending the name attaches to that family name and is written
     before it — provided at least one given word remains, so that a
     name whose only given word is the particle keeps it (the
-    words-to-spare test S2 applies to ambiguous suffixes). Where
+    words-to-spare test S2 applies to ambiguous suffixes). A
+    post-nominal is written BEHIND the particle in this listing, so
+    it does not end the name for this purpose: the run is found by
+    looking past trailing words that hold no name — unless such a
+    word is itself particle vocabulary, which makes it part of the
+    run rather than something to look past. Where
     the word is BOTH a particle and suffix vocabulary, this
     attachment outranks the suffix reading (S2): a trailing
     abbreviation after a family comma is the tussenvoegsel far more
     often than the decoration it collides with.
-      "Jong, Anke de"             →  family="de Jong"  deviates: #379 (today: family="Jong")
-      "Beethoven, Ludwig van"     →  family="van Beethoven"  deviates: #379 (today: family="Beethoven")
-      "Berg, Jan vd"              →  family="vd Berg"  deviates: #380 (today: family="Berg")
+      "Jong, Anke de"             →  family="de Jong"
+      "Beethoven, Ludwig van"     →  family="van Beethoven"
+      "Berg, Jan vd"              →  family="vd Berg"
+      "Berg, Jan van der"         →  family="van der Berg"
+      "Vega, Juan de la"          →  family_particles="de la"
+      "Beethoven, Ludwig van"     →  family_base="Beethoven"
+      "Beethoven, Ludwig van"     →  family_particles="van"
       "Nguyen, Van"               →  given="Van"  · boundary
+    Accepted: an ambiguous particle attaches on the same terms as a
+    never-given one, so a Vietnamese name written in this listing
+    loses its given name — but only in the UNACCENTED
+    transliteration. Vân carries a diacritic and is not particle
+    vocabulary, so the correctly spelled name never reaches this
+    rule. It is the ASCII spelling that collides, and there the two
+    traditions write the same string.
+      "Nguyen, Thi Van"           →  family="Van Nguyen"
+      "Nguyễn, Thị Vân"           →  family="Nguyễn"
+    Accepted: no ambiguity is reported for that collision, which A1
+    would call for. The fork is decided here, and assign's emitter is
+    scoped to the no-comma shapes on the reasoning that a comma has
+    fixed the family — true of the family, not of the particle behind
+    it. Tracked at #405.
+    Accepted: the colliding spelling has a format that reads
+    correctly, and it is ONE order, not both: FAMILY_FIRST still
+    sends the given name to the middle, and only
+    FAMILY_FIRST_GIVEN_LAST recovers it.
+      "Nguyen Thi Van"  family-first-given-last  →  family="Nguyen"
+      "Nguyen Thi Van"  family-first-given-last  →  given="Van"
+      "Nguyen Thi Van"  family-first  →  middle="Van"
     Accepted: without a family comma the name's written shape is not
     settled — "Jong Anke de" may be a misformatted listing, and a
     bare "Jong de" may be a given name beside a particle — so the
@@ -272,9 +302,11 @@ P6. Rationale: a particle ending the name has nothing to link
     Accepted: the precedence over S2 is stated for the shape, so it
     sweeps in every word that is both particle and suffix
     vocabulary — today vd, do and mc. Only vd's reading was
-    weighed; the other two inherit it, which is the shape's cost
-    and is tracked with the other contested memberships.
-    history: decisions.md#P6 · interacts: C1, P1, S2 · tracked: #379, #380
+    weighed; mc inherits it, which is the shape's cost and is
+    tracked with the other contested memberships. `do` sits in the
+    AMBIGUOUS acronym half and was already read as a name word
+    there, so the precedence decides nothing for it.
+    history: decisions.md#P6 · interacts: C1, P1, S2 · implemented: nameparser/_pipeline/_post_rules.py
 
 ## Suffixes: generational & credentials (S)
 
@@ -660,8 +692,9 @@ R1. Rationale: a field is a way of reading the parse, not a stored
     string.
     Every field is a view computed from the parsed words at read
     time, joining its words in written order — except folded family
-    words (O3), which render before the rest of the family wherever
-    they stood in the string.
+    words — O3's fold and, since #379, P6's attached tussenvoegsel —
+    which render before the rest of the family wherever they stood in
+    the string.
       "Dr. Juan Q. Xavier de la Vega III"  →  family="de la Vega"
       "Hassan, Mohamad Ahmad Ali"  middle_as_family  →  family="Ahmad Ali Hassan"
       "Hassan, Mohamad Ahmad Ali"          →  family="Hassan"  · boundary

docs/release_log.rst

@@ -32,6 +32,8 @@ Release Log
 
     - Fix a name opening with a particle that is *never* a given name being split at the particle under a family-first name order -- ``Policy(name_order=FAMILY_FIRST)`` and ``Policy(name_order=FAMILY_FIRST_GIVEN_LAST)`` alike, and identically: ``"de Mesnil"`` read as family ``de``, given ``Mesnil``, and ``"de la Vega"`` as family ``de``, given ``la Vega``. Each is now the whole surname, as it has always been in the default order. The rule enforcing it asked for the particle by the ``GIVEN`` role, which under a family-first order belongs to the token *after* the particle, so the test read the wrong word and declined. It now also asks by position -- the piece that opens the name -- so both shapes of the same rule are caught: where such a particle stands alone as a piece, either opening the name or in the given position, the name is left with no given name at all, the given and the middles folding into the family. Standing *alone* is the whole of it, and the rule claims nothing wider: ``"Juan de la Vega"`` under ``FAMILY_FIRST`` still reports given ``de la Vega``, because there the particle chained onto the words after it rather than standing alone, and a bare ``"de"`` with nothing to fold into is still reported as the given name. The decision behind the fix: a word that can never be a given name leaves ``name_order`` nothing to decide, so declaring family-first is not a reason to make ``de`` a surname on its own. A leading particle that *may* be a given name is genuinely order-dependent and is untouched -- ``"van Gogh"`` still reads as family ``van``, given ``Gogh`` under both family-first orders. This is also what gives ``Lexicon.particles_ambiguous`` an effect outside the default order: taking a word out of it now changes the parsed fields under a family-first order, where before it moved only the ambiguity report. Seven of the 751 differential corpus names move, the same seven under each family-first order; default-order output is byte-identical over all 751, at the 1.4.0, 2.0.0 and 2.1.0 differential baselines alike (closes #359)
 
+    - Fix a tussenvoegsel after a family comma being parsed as a middle name. Dutch and Belgian alphabetized listings move the particle behind the given name -- ``"Beethoven, Ludwig van"`` is how ``"Ludwig van Beethoven"`` is filed -- and the trailing particle run was read as a middle name rather than as part of the surname: ``"Beethoven, Ludwig van"`` gave middle ``van``, last ``Beethoven``, and ``"Berg, Jan van der"`` gave middle ``van der``. The run now attaches to the family the comma has already named and renders before it, so those read family ``van Beethoven`` and ``van der Berg`` with the given name unchanged. The derived views move with the parse, so ``family_particles`` is ``van`` and ``family_base`` is ``Beethoven`` where they were empty and ``Beethoven`` before. `#130 <https://github.com/derek73/python-nameparser/issues/130>`_ asked for the split and got it in 1.3.0 as ``last_base``/``last_prefixes``; 2.0 renamed them ``family_base``/``family_particles``. What was wrong until now was the values they reported for this listing. Both halves of the particle vocabulary attach -- never-given ``de`` and may-be-given ``van`` alike -- because after a comma the family is already named and the particle has no other role to take. Two guards bound it. A name whose only given word is the particle keeps it, so ``"Nguyen, Van"`` still reads given ``Van``: the attachment needs a given word to spare. And where the word is BOTH particle and suffix vocabulary the attachment outranks the post-nominal reading, so ``"Berg, Jan vd"`` reads family ``vd Berg`` where 1.4.0 and 2.1 alike gave suffix ``vd`` -- a trailing abbreviation after a family comma is the tussenvoegsel far more often than the decoration it collides with, and the same shape sweeps in ``mc``, which 2.1 also read as a suffix. ``do`` is in ``SUFFIX_ACRONYMS_AMBIGUOUS`` and 2.1 already read a trailing one as a name word, so it attaches by the plain rule rather than by the override (closes #379, closes #380). Names without the comma are untouched: ``"Ludwig van Beethoven"`` already read family ``van Beethoven`` and is byte-identical. One of the 751 differential corpus names moves, ``"Vega, Juan de la"``, at the 1.4.0, 2.0.0 and 2.1.0 baselines alike
+
     - Add ``abd`` to :data:`~nameparser.config.bound_given_names.BOUND_GIVEN_NAMES`, so the spellings that write the article as its own word join like the others do: ``"abd Allah Smith"`` was given ``abd``, middle ``Allah`` and is now given ``abd Allah``. ``abdul``, ``abdel`` and ``abdal`` were already there, and the Arabic-script ``عبد`` has covered the same word since 2.0, so only the Latin spelling was short. The word is also the postnominal ABD ("All But Dissertation") and stays in ``SUFFIX_ACRONYMS``: position tells the two readings apart, so ``"Jane Smith ABD"``, ``"Jane Smith, ABD"`` and ``"Jane Smith A.B.D."`` all still read the credential as a suffix. Making that work needed one fix beyond the wordlist. The join reserves enough words to leave a family name behind, and the reserve counted every piece except titles and suffixes -- so a word in BOTH vocabularies did not count as a name word even when the rule had already claimed it as one, and the join declined silently on the common three-word shape. The claimed piece now counts, which is what the reserve means: it asks whether enough OTHER words are left to spare. No shipped word other than ``abd`` is in both sets, and for any word that is not, the changed expression reduces to the old one -- so nothing else can move, by arithmetic rather than by sampling. The differential corpus agrees at all 751 names, but it holds no name carrying a dual-membership word, so that agreement tests the corpus rather than the change (#400)
 
     - Change how far a leading never-given particle takes the surname when a family-first ``name_order`` is declared. ``Policy(name_order=FAMILY_FIRST)`` read ``"de Mesnil Juan"`` as family ``de Mesnil Juan`` -- the whole name -- and now reads family ``de Mesnil``, given ``Juan``. Declaring a family-first order asserts that what follows the family is not more surname, and where the surname run ends is exactly that question, so the declaration settles it. The default order is unchanged, deliberately: with no order declared nothing marks where the surname ends, and a particle followed by several words really can be all surname -- ``von Bergen Wessels`` is one such name. Nothing in ``"de Mesnil Juan"`` distinguishes it from that reading except a declared order or a comma. A caller who means family ``de la Vega`` plus given ``Juan`` in the default order writes the comma, which already parses that way. The run takes one name WORD rather than one token: a conjunction-joined run and a bound given-name pair each count once, so the stop cannot land inside one -- ``"de la Vega y Santos Juan"`` reads family ``de la Vega y Santos``, and ``"ibn Awf abdul Rahman"`` reads given ``abdul Rahman``. Where two or more words are left over the two family-first orders differ from each other for the first time: ``"de la Cruz Juan Carlos"`` reads given ``Juan``, middle ``Carlos`` under ``FAMILY_FIRST`` and middle ``Juan``, given ``Carlos`` under ``FAMILY_FIRST_GIVEN_LAST``. An ambiguous leading particle is untouched in every order -- ``"van Gogh Jan Pieter"`` still reads family ``van`` under both family-first orders -- and so is a family comma, where the comma has already fixed the surname (``"Smith, de Mesnil"`` keeps family ``Smith de Mesnil``). One of the 751 differential corpus names moves, ``"de Mesnil Garcia"`` to family ``de Mesnil``, given ``Garcia``, under each family-first order; default-order output is byte-identical over all 751. This reverses the answer #364 was closed on, and the reasoning is recorded at ``docs/design/decisions.md#P1`` (closes #395)