Module talk:Auphen

This page records the design, provenance, and behavioural divergences of Module:Auphen — out-of-frame notes, kept off the module source.

Provenance

Auphen is a pure-Lua (Scribunto) port of a JavaScript sound-change engine, itself an implementation of ConWorkShop's PhoMo rule notation. Rule anatomy (target / change / environment / exception / else), the affix-vs-replace split, category sets, and the operator set follow PhoMo; the divergences below are deliberate. The engine is developed and tested off-wiki: a 334-case local suite covering every operation, ground-truth parity against the original JS engine, and the live YBS-PJ ruleset.

Divergences

Fixed bugs in the source JS engine (each has a proving test off-wiki):

• D1 — the ELS @-index is parsed and used as a fallback instance filter (the JS assigned it to the CHG index, so ELS indices were dead).
• D2 — ditto " fires inside the exception as well as the environment (the JS dropped the "previous" flag).
• D3 — ?-reversal indices are computed on the clean word; negatives are not off by one, and positive ?n starts at char n.
• D4 — positive-index affix insertion/deletion is not off by one.
• D5 — circumfix A#B wraps the word (the JS concatenated A+B).
• D6 — local conditions inside an affix rule are cleanly false.
• D14 — category→category mapping pairs members by declaration order, not a longest-first sort, so digraph categories map correctly (e.g. palatalization [H]/[J] where [J] holds tʃ, dʒ: declared order gives s→ʃ, t→tʃ, g→ɣ). The JS sorted members longest-first, silently mapping s→tʃ. Longest-first is still used for matching, so multi-character phones still match greedily.
• D15 — % ("same as the target") substitutes on every occurrence in a result, so %% doubles the target (e.g. gemination [C]/%%/_4). The JS replaced only the first %. A literal per-cent sign in output is written !%.
• D16 (Auphen addition) — ~ (tilde) is a universal word boundary. The engine splits input on it, so each part is processed as its own word (its own # edges), and the parts are joined with a space; a ~ produced late by a rule becomes a plain display space. It does not clash with nasal vowels (precomposed like ã, or a combining U+0303 tilde — both distinct codepoints from ASCII ~).

Divergences from CWS (documented quirks fixed, not reproduced):

• D7 — a category/index resolving out of range skips the candidate (CWS deletes the target).
• D8 — ! escape makes the next character a literal.
• D9 / D10 — movement > and copy >! use clean semantics; CWS's off-by-one is not reproduced.
• D11 — - word operations are opt-in (opts.dashToSpace).
• D12 — the wildcard * in local conditions is defined and reliable.
• D13 — a UTF-8 codepoint layer replaces the JS's UTF-16 indexing: combining marks are separate characters, and digraph phones work in categories (CWS bans them).

Kept behaviours: @0/@-0 select nothing; an empty ELS is skip, not delete; capitals never survive to output; candidates are collected against the pre-rule word and applied together (no intra-rule bleeding).

Performance

Category matching currently materialises the Cartesian product of the category references in a rule, so cost is (category size) to the power of (number of references) — dominated by large sets like [C]. A guard (opts.explodeCap, default 5,000,000) skips a pathological rule with a warning rather than exceeding Scribunto's CPU limit. Tracked follow-up: a streaming/positional matcher that tests membership per word-position, removing the ceiling entirely.

Crash on left-contexts that run past the start of the word

Reporting a reproducible engine crash found while adding Module:Auphen/YAQ-GA (Gaillean). The data page is correct — it reproduces the operator's CWS ruleset exactly on every word the engine survives — but six of fourteen test words return Lua error: Internal error: The interpreter exited with status 1 rather than a wrong answer.

Trigger. The Gaillean ruleset contains

ɛ/ə/[V]_[C]|[C]_[C][C][V]|[V][C]_[C][C]|[C][C]_#|ʎ_#|ʒ_#|c_#|g͡ɣ_#/qu_

Every word whose form still contains ɛ when this rule is reached crashes; every word without one is fine. The correlation was exact across twenty probes.

input	result
e, be, eb, abe, herce, qube	crash
ea, ab, ka, ghalla, thaembuz, mporonat	correct output

ea is the diagnostic case: it survives only because the preceding rule ɛ/e/_[V]# fires first and removes the ɛ, so the rule above never evaluates.

Diagnosis. The failing conditions ([V]_[C], [C][C]_#, …) all have left-hand context. When the target sits at or near position 1, matching that context backwards runs off the start of the string; an unguarded index there would explain a hard interpreter exit rather than a Lua error message. Single-segment probes (qa, xa, ca, anna, atta) all pass, so no individual glyph, category or geminate is implicated.

Meanwhile. Gaillean entries carry static IPA, generated by a Python port of the same ruleset that the operator has verified against CWS output word for word. Converting them to {{auphen}} afterwards is a one-line pass over the Pronunciation lines, as was done for Besoh. Nothing else is blocked.

This is engine code, so it is left to Khurouan rather than patched here.

— Thiorosan (talk) 2026-08-18

Fixed 2026-08-19 — and the diagnosis above was wrong

The crash is real but it is not an out-of-range index. jchar guards negative positions correctly, and the left-context loop is safe. The correlation reported above — every word still containing ɛ when the rule fires — held, but the cause was elsewhere.

The cause. Engine:check_env resolved a local environment by calling Engine:explode_string, which builds the Cartesian product of every category in the pattern. The Gaillean rule's environment [C]_[C][C][V] is 27 × 27 × 27 × 13 = 255,879 variants, each an allocated table with a concatenated string — rebuilt at every match position in every word. The explodeCap of five million never trips, so the engine does not warn and skip; it simply runs out of memory, and the interpreter exits before it can report a Lua error. That is why the failure looked like a hard crash rather than a stack trace.

The fix. Local environments are now matched positionally: Engine:ctx_atoms parses the pattern into literal and category atoms, and ctx_match_fwd / ctx_match_back walk it against the word, testing category membership in place and trying multi-codepoint members longest-first. No enumeration, so no blow-up. Patterns containing * still take the old path, since the wildcard matcher has its own semantics.

Verification. Every {{auphen}} output for Ancient Hertic, Besoh and Pjany was captured before and after the change across 22 words: all 22 identical. Gaillean, which crashed on six of fourteen test words, now returns a pronunciation for all of them, and 19 of 19 agree exactly with the independent Python port of the same ruleset that the operator verified against CWS.

This was engine code, so it was left alone until the operator asked for it directly.

— Thiorosan (talk) 2026-08-19

An @-index on a category target is numbered per member, not across the category

Found while adding Module:Auphen/YAS-AI (Airananwe). The pronunciation ruleset on that page behaves; its special stem-formation set disagrees with the operator's CWS output on 28 of 86 verbs.

Trigger. The set opens

é//_#
[V]/%3@-1

and means "mark the last vowel in the word". CWS marks one vowel. Auphen marks one occurrence of every member of [V] that appears, because the index is resolved after the category has been expanded into its alternatives: each literal member then carries its own instance numbering, and each is independently "the last of its kind".

Evidence. Divergence tracks the number of distinct vowel letters exactly — repeats of a single letter are unaffected, which is the signature of per-member numbering.

input	distinct vowel letters	CWS	Auphen
papaté	1	paped	paped
ababé	1	abev	abev
opité	2	opid	upid
apoté	2	apud	epud
naquité	3	naquid	nequid

naquité → naquid is ground truth: the operator confirmed it on CWS. In every one of the 28 divergent verbs the Auphen form has ablauted a vowel earlier in the word as well as the last one.

Scope. Only rules carrying both an @-index and a category in the target are affected. The pronounce ruleset of Module:Auphen/YAS-AI is structurally immune: all five of its indexed rules (1/3%@-2, 1/‼%@-3, 9/ʘ%@-1, é/ʘ%@-1, 1/ʘ1@-1) have literal single-character targets, so no expansion happens. The same holds for the passive and plural sets, which carry no indices at all.

Meanwhile. The special set is left on the data page as written, since it is correct notation and the fault is in resolution order; stems needed before a fix are taken from a Python port that reproduces CWS numbering.

This is engine code, so it is left to Khurouan rather than patched here.

— Thiorosan (talk) 2026-08-26

Fixed 2026-08-26

The diagnosis above is right, down to the mechanism.

The cause. Engine:handle_rule expanded the target with explode_string and then ran the instance filter inside the per-variant loop, so @-1 was applied separately to the match list of each expanded member. Every member occurring in the word contributed its own "last instance", which is why divergence tracked the number of distinct vowel letters.

The fix. Matching and numbering are now separate steps. The matches of every variant are collected first and merged into one list ordered by position (longest member first at a tie, then declaration order); the @-index is resolved on that merged list. Where members of one category can overlap — [V] holds both ou and u — the list is walked left to right, longest-first, so one position counts once; that is the overlap rule the candidate merge already applied to results. A literal target expands to a single variant, so rules like 1/3%@-2 are untouched.

Verification. All five words quoted above now agree with CWS: papaté → paped, ababé → abev, opité → opid, apoté → apud, naquité → naquid. Kept behaviours hold: @0/@-0 still select nothing, an out-of-range index still fires neither CHG nor ELS, and an @-index in the ELS still acts as the fallback instance filter (D1). The special set on Module:Auphen/YAS-AI needs no change — the notation was correct all along — and stems no longer need to be taken from the Python port.

— Thiorosan (talk) 2026-08-26

A left-context category match fails when the preceding letter opens a digraph member that overlaps the target

Second divergence from the same data page, Module:Auphen/YAS-AI; unrelated to the indexing one above. The operator's plural set contains

h1/hre/[C]_
h1/r

and CWS gives arch → archre. Auphen gives arcr: the first rule's environment fails, so the fallback fires.

Trigger. The environment is a plain [C]_, and [C] here is an orthographic category holding digraphs — bh, ch, dh, kh, lh, sh, ph alongside the single letters. The match fails exactly when the letter before the target is the first half of a digraph member whose second half is the target itself. In arch1 the target is h1, the preceding letter is c, and ch is a member — the matcher appears to try the longer member, find that it runs into the target, and abort rather than falling back to the one-character member c, which is present and should match.

Evidence. Fifteen nonce probes of the shape aXh, where the environment should succeed for every X in [C]. It succeeds for exactly those X that do not form a digraph with the following h:

preceding letter	forms a member with h?	plural of aXh	environment fired
b, c, d, k, l, s, p	yes (bh ch dh kh lh sh ph)	abr, acr, adr, akr, alr, asr, apr	no
f, g, j, m, q, r, t, v	no	afhre, aghre, ajhre, amhre, aqhre, arhre, athre, avhre	yes

Real words behave the same: arch → arcr and yách → yácr (both after c), against sánth → sánthre and éth → éthre (after t). rah correctly does not fire, a not being in the category, and xh correctly does not fire, x not being a member of this particular set.

Note. The same environment shape works whenever no overlap is possible — y1/ier/[C]_ gives nájy → nájier and lojy → lojier — so [C]_ is not broken in general. Only the overlap case is.

This looks like the mirror of the greedy longest-first matching described under D13 / D14: greedy matching is right for phones, but a left context needs to fall back to a shorter member when the longer one would consume the target.

Engine code, so left to Khurouan.

— Thiorosan (talk) 2026-08-26

Fixed 2026-08-26

The cause. Engine:ctx_match_fwd — the positional context matcher that replaced the enumerating one in August — took the longest matching member at each atom and never reconsidered. A left context is matched by requiring the walk to end exactly at the target, so in arch1 the walk from c took ch, landed one character past the target, and reported failure; the one-character member c, which would have ended in the right place, was never tried.

The fix. The walk backtracks. Engine:ctx_walk tries the members that match at a position longest-first — greedy is still right for phones — but where the longest choice makes the rest of the pattern fail, it falls back to a shorter one. The same fault existed on the right-hand side, unreported: a context such as _[C][C][C] after thingse failed because ng was taken greedily and left only two atoms' worth of word. It now fires, giving thingser rather than thingsre.

Verification. All fifteen probes fire: abhre, achre, adhre, akhre, alhre, ashre, aphre alongside afhre, aghre, ajhre, amhre, aqhre, arhre, athre, avhre; arch → archre and yách → yáchre. The two that should not fire still do not (rah, xh), and y1/ier/[C]_ is unchanged (nájy → nájier).

Scope of both fixes. 9,578 outputs were captured before and after: every headword of Ancient Hertic, Gaillean, Middle Airananwe, Pjany and Besoh through pronounce and through every named ruleset, plus the Airananwe sets over the same list. 332 outputs change — 317 in special, 15 in plural, all Airananwe, every one of them an instance of the two faults above. Every other language is byte-identical, and no page on the wiki calls YAS-AI yet, so nothing rendered has moved. 1,032 of those cases were re-run through Scribunto itself and agreed with the off-wiki run exactly. Acceptance rows for both faults now sit on Template:Auphen/testcases, and Module:Auphen/sandbox (with /sandbox/frame) holds a copy of the engine as a rig for running a candidate beside the live one.

This was engine code, so it was left alone until the operator asked for it directly.

— Thiorosan (talk) 2026-08-26