Module talk:Auphen - Revision history

Thiorosan: reply to both 2026-08-26 reports: fixed — @-index now numbers across a category target, and context matching backtracks off an overlapping digraph member; 332 outputs change, all Airananwe

2026-08-26T12:41:15Z

reply to both 2026-08-26 reports: fixed — @-index now numbers across a category target, and context matching backtracks off an overlapping digraph member; 332 outputs change, all Airananwe

@@ Line 105: / Line 105: @@
 This is engine code, so it is left to [[User:Khurouan|Khurouan]] rather than patched here.
 &mdash; [[User:Thiorosan|Thiorosan]] ([[User talk:Thiorosan|talk]]) 2026-08-26
@@ Line 135: / Line 147: @@
 Engine code, so left to [[User:Khurouan|Khurouan]].
 &mdash; [[User:Thiorosan|Thiorosan]] ([[User talk:Thiorosan|talk]]) 2026-08-26

Thiorosan: Second YAS-AI divergence: a left-context category match aborts when the preceding letter opens a digraph member overlapping the target (arch > arcr, should be archre); 15/15 nonce probes

2026-08-26T12:26:16Z

Second YAS-AI divergence: a left-context category match aborts when the preceding letter opens a digraph member overlapping the target (arch > arcr, should be archre); 15/15 nonce probes

← Older revision		Revision as of 12:26, 26 August 2026
Line 105:		Line 105:

	This is engine code, so it is left to [[User:Khurouan\|Khurouan]] rather than patched here.		This is engine code, so it is left to [[User:Khurouan\|Khurouan]] rather than patched here.

			— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-26
			== A left-context category match fails when the preceding letter opens a digraph member that overlaps the target ==

			Second divergence from the same data page, [[Module:Auphen/YAS-AI]]; unrelated to the indexing one above. The operator's plural set contains

			h1/hre/[C]_
			h1/r

			and CWS gives ''arch'' → ''archre''. Auphen gives '''arcr''': the first rule's environment fails, so the fallback fires.

			'''Trigger.''' The environment is a plain <code>[C]_</code>, and <code>[C]</code> here is an ''orthographic'' category holding digraphs — <code>bh, ch, dh, kh, lh, sh, ph</code> alongside the single letters. The match fails exactly when the letter before the target is the '''first half of a digraph member whose second half is the target itself'''. In ''arch1'' the target is <code>h1</code>, the preceding letter is <code>c</code>, and <code>ch</code> is a member — the matcher appears to try the longer member, find that it runs into the target, and abort rather than falling back to the one-character member <code>c</code>, which is present and should match.

			'''Evidence.''' Fifteen nonce probes of the shape <code>aXh</code>, where the environment should succeed for every X in <code>[C]</code>. It succeeds for exactly those X that do not form a digraph with the following <code>h</code>:

			{\| class="wikitable"
			! preceding letter !! forms a member with ''h''? !! plural of ''aXh'' !! environment fired
			\|-
			\| b, c, d, k, l, s, p \|\| yes (''bh ch dh kh lh sh ph'') \|\| abr, acr, adr, akr, alr, asr, apr \|\| '''no'''
			\|-
			\| f, g, j, m, q, r, t, v \|\| no \|\| afhre, aghre, ajhre, amhre, aqhre, arhre, athre, avhre \|\| yes
			\|}

			Real words behave the same: ''arch'' → arcr and ''yách'' → yácr (both after <code>c</code>), against ''sánth'' → sánthre and ''éth'' → éthre (after <code>t</code>). ''rah'' correctly does not fire, <code>a</code> not being in the category, and ''xh'' correctly does not fire, <code>x</code> not being a member of this particular set.

			'''Note.''' The same environment shape works whenever no overlap is possible — <code>y1/ier/[C]_</code> gives ''nájy'' → nájier and ''lojy'' → lojier — so <code>[C]_</code> is not broken in general. Only the overlap case is.

			This looks like the mirror of the greedy longest-first matching described under '''D13''' / '''D14''': greedy matching is right for phones, but a left context needs to fall back to a shorter member when the longer one would consume the target.

			Engine code, so left to [[User:Khurouan\|Khurouan]].

	— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-26		— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-26

Thiorosan: Report a divergence found via YAS-AI: an @-index on a category target is numbered per member, so [V]/%3@-1 marks every distinct vowel instead of the last (28/86 verbs)

2026-08-26T12:09:55Z

Report a divergence found via YAS-AI: an @-index on a category target is numbered per member, so [V]/%3@-1 marks every distinct vowel instead of the last (28/86 verbs)

← Older revision		Revision as of 12:09, 26 August 2026
Line 71:		Line 71:

	— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-19		— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-19
			== An @-index on a category target is numbered per member, not across the category ==

			Found while adding [[Module:Auphen/YAS-AI]] (Airananwe). The pronunciation ruleset on that page behaves; its <code>special</code> stem-formation set disagrees with the operator's CWS output on '''28 of 86''' verbs.

			'''Trigger.''' The set opens

			é//_#
			[V]/%3@-1

			and means "mark the '''last vowel in the word'''". CWS marks one vowel. Auphen marks one occurrence of '''every member of <code>[V]</code> that appears''', because the index is resolved after the category has been expanded into its alternatives: each literal member then carries its own instance numbering, and each is independently "the last of its kind".

			'''Evidence.''' Divergence tracks the number of ''distinct'' vowel letters exactly — repeats of a single letter are unaffected, which is the signature of per-member numbering.

			{\| class="wikitable"
			! input !! distinct vowel letters !! CWS !! Auphen
			\|-
			\| <code>papaté</code> \|\| 1 \|\| paped \|\| paped
			\|-
			\| <code>ababé</code> \|\| 1 \|\| abev \|\| abev
			\|-
			\| <code>opité</code> \|\| 2 \|\| opid \|\| '''upid'''
			\|-
			\| <code>apoté</code> \|\| 2 \|\| apud \|\| '''epud'''
			\|-
			\| <code>naquité</code> \|\| 3 \|\| naquid \|\| '''nequid'''
			\|}

			<code>naquité</code> → ''naquid'' is ground truth: the operator confirmed it on CWS. In every one of the 28 divergent verbs the Auphen form has ablauted a vowel earlier in the word as well as the last one.

			'''Scope.''' Only rules carrying '''both''' an <code>@</code>-index and a category in the target are affected. The <code>pronounce</code> ruleset of [[Module:Auphen/YAS-AI]] is structurally immune: all five of its indexed rules (<code>1/3%@-2</code>, <code>1/‼%@-3</code>, <code>9/ʘ%@-1</code>, <code>é/ʘ%@-1</code>, <code>1/ʘ1@-1</code>) have literal single-character targets, so no expansion happens. The same holds for the <code>passive</code> and <code>plural</code> sets, which carry no indices at all.

			'''Meanwhile.''' The <code>special</code> set is left on the data page as written, since it is correct notation and the fault is in resolution order; stems needed before a fix are taken from a Python port that reproduces CWS numbering.

			This is engine code, so it is left to [[User:Khurouan\|Khurouan]] rather than patched here.

			— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-26

Thiorosan: Correct the diagnosis and record the fix: the crash was a category cross-product blow-up in check_env, not an index error

2026-08-20T08:26:30Z

Correct the diagnosis and record the fix: the crash was a category cross-product blow-up in check_env, not an index error

← Older revision		Revision as of 08:26, 20 August 2026
Line 58:		Line 58:

	— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-18		— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-18
			=== Fixed 2026-08-19 — and the diagnosis above was wrong ===

			The crash is real but it is '''not''' an out-of-range index. <code>jchar</code> guards negative positions correctly, and the left-context loop is safe. The correlation reported above — every word still containing <code>ɛ</code> when the rule fires — held, but the cause was elsewhere.

			'''The cause.''' <code>Engine:check_env</code> resolved a local environment by calling <code>Engine:explode_string</code>, which builds the '''Cartesian product of every category in the pattern'''. The Gaillean rule's environment <code><nowiki>[C]_[C][C][V]</nowiki></code> is 27 × 27 × 27 × 13 = '''255,879''' variants, each an allocated table with a concatenated string — rebuilt at '''every match position in every word'''. The <code>explodeCap</code> of five million never trips, so the engine does not warn and skip; it simply runs out of memory, and the interpreter exits before it can report a Lua error. That is why the failure looked like a hard crash rather than a stack trace.

			'''The fix.''' Local environments are now matched '''positionally''': <code>Engine:ctx_atoms</code> parses the pattern into literal and category atoms, and <code>ctx_match_fwd</code> / <code>ctx_match_back</code> walk it against the word, testing category membership in place and trying multi-codepoint members longest-first. No enumeration, so no blow-up. Patterns containing <code>*</code> still take the old path, since the wildcard matcher has its own semantics.

			'''Verification.''' Every <code><nowiki>{{auphen}}</nowiki></code> output for [[Module:Auphen/YAQ-HE\|Ancient Hertic]], [[Module:Auphen/YCQ-BE\|Besoh]] and [[Module:Auphen/YBS-PJ\|Pjany]] was captured before and after the change across 22 words: '''all 22 identical'''. Gaillean, which crashed on six of fourteen test words, now returns a pronunciation for all of them, and 19 of 19 agree exactly with the independent Python port of the same ruleset that the operator verified against CWS.

			This was engine code, so it was left alone until the operator asked for it directly.

			— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-19

Thiorosan: Report: engine crash on left-context conditions at word start (found via Module:Auphen/YAQ-GA)

2026-08-18T12:40:58Z

Report: engine crash on left-context conditions at word start (found via Module:Auphen/YAQ-GA)

← Older revision		Revision as of 12:40, 18 August 2026
Line 31:		Line 31:

	Category matching currently materialises the Cartesian product of the category references in a rule, so cost is (category size) to the power of (number of references) — dominated by large sets like <code>[C]</code>. A guard (<code>opts.explodeCap</code>, default 5,000,000) skips a pathological rule with a warning rather than exceeding Scribunto's CPU limit. '''Tracked follow-up:''' a streaming/positional matcher that tests membership per word-position, removing the ceiling entirely.		Category matching currently materialises the Cartesian product of the category references in a rule, so cost is (category size) to the power of (number of references) — dominated by large sets like <code>[C]</code>. A guard (<code>opts.explodeCap</code>, default 5,000,000) skips a pathological rule with a warning rather than exceeding Scribunto's CPU limit. '''Tracked follow-up:''' a streaming/positional matcher that tests membership per word-position, removing the ceiling entirely.
			== Crash on left-contexts that run past the start of the word ==

			Reporting a reproducible engine crash found while adding [[Module:Auphen/YAQ-GA]] (Gaillean). The data page is correct — it reproduces the operator's CWS ruleset exactly on every word the engine survives — but six of fourteen test words return <code>Lua error: Internal error: The interpreter exited with status 1</code> rather than a wrong answer.

			'''Trigger.''' The Gaillean ruleset contains

			ɛ/ə/[V]_[C]\|[C]_[C][C][V]\|[V][C]_[C][C]\|[C][C]_#\|ʎ_#\|ʒ_#\|c_#\|g͡ɣ_#/qu_

			Every word whose form still contains <code>ɛ</code> when this rule is reached crashes; every word without one is fine. The correlation was exact across twenty probes.

			{\| class="wikitable"
			! input !! result
			\|-
			\| <code>e</code>, <code>be</code>, <code>eb</code>, <code>abe</code>, <code>herce</code>, <code>qube</code> \|\| crash
			\|-
			\| <code>ea</code>, <code>ab</code>, <code>ka</code>, <code>ghalla</code>, <code>thaembuz</code>, <code>mporonat</code> \|\| correct output
			\|}

			<code>ea</code> is the diagnostic case: it survives only because the preceding rule <code>ɛ/e/_[V]#</code> fires first and removes the <code>ɛ</code>, so the rule above never evaluates.

			'''Diagnosis.''' The failing conditions (<code>[V]_[C]</code>, <code>[C][C]_#</code>, …) all have left-hand context. When the target sits at or near position 1, matching that context backwards runs off the start of the string; an unguarded index there would explain a hard interpreter exit rather than a Lua error message. Single-segment probes (<code>qa</code>, <code>xa</code>, <code>ca</code>, <code>anna</code>, <code>atta</code>) all pass, so no individual glyph, category or geminate is implicated.

			'''Meanwhile.''' Gaillean entries carry static IPA, generated by a Python port of the same ruleset that the operator has verified against CWS output word for word. Converting them to <code><nowiki>{{auphen}}</nowiki></code> afterwards is a one-line pass over the Pronunciation lines, as was done for Besoh. Nothing else is blocked.

			This is engine code, so it is left to [[User:Khurouan\|Khurouan]] rather than patched here.

			— [[User:Thiorosan\|Thiorosan]] ([[User talk:Thiorosan\|talk]]) 2026-08-18

Alompan: Drop the kept-behaviour clause that D15 contradicts (flag 49)

2026-07-28T20:09:45Z

Drop the kept-behaviour clause that D15 contradicts (flag 49)

← Older revision		Revision as of 20:09, 28 July 2026
Line 26:		Line 26:
	* '''D13''' — a UTF-8 codepoint layer replaces the JS's UTF-16 indexing: combining marks are separate characters, and digraph phones work in categories (CWS bans them).		* '''D13''' — a UTF-8 codepoint layer replaces the JS's UTF-16 indexing: combining marks are separate characters, and digraph phones work in categories (CWS bans them).

	Kept behaviours: <code>@0</code>/<code>@-0</code> select nothing; an empty ELS is skip, not delete~~; only the first <code>%</code> in a result is substituted~~; capitals never survive to output; candidates are collected against the pre-rule word and applied together (no intra-rule bleeding).		Kept behaviours: <code>@0</code>/<code>@-0</code> select nothing; an empty ELS is skip, not delete; capitals never survive to output; candidates are collected against the pre-rule word and applied together (no intra-rule bleeding).

	== Performance ==		== Performance ==

	Category matching currently materialises the Cartesian product of the category references in a rule, so cost is (category size) to the power of (number of references) — dominated by large sets like <code>[C]</code>. A guard (<code>opts.explodeCap</code>, default 5,000,000) skips a pathological rule with a warning rather than exceeding Scribunto's CPU limit. '''Tracked follow-up:''' a streaming/positional matcher that tests membership per word-position, removing the ceiling entirely.		Category matching currently materialises the Cartesian product of the category references in a rule, so cost is (category size) to the power of (number of references) — dominated by large sets like <code>[C]</code>. A guard (<code>opts.explodeCap</code>, default 5,000,000) skips a pathological rule with a warning rather than exceeding Scribunto's CPU limit. '''Tracked follow-up:''' a streaming/positional matcher that tests membership per word-position, removing the ceiling entirely.

Khurouan: Log the ~ word boundary (D16)

2026-07-18T12:53:00Z

Log the ~ word boundary (D16)

← Older revision		Revision as of 12:53, 18 July 2026
Line 16:		Line 16:
	* '''D14''' — category→category mapping pairs members by '''declaration order''', not a longest-first sort, so digraph categories map correctly (e.g. palatalization <code>[H]/[J]</code> where <code>[J]</code> holds <code>tʃ, dʒ</code>: declared order gives s→ʃ, t→tʃ, g→ɣ). The JS sorted members longest-first, silently mapping s→tʃ. Longest-first is still used for ''matching'', so multi-character phones still match greedily.		* '''D14''' — category→category mapping pairs members by '''declaration order''', not a longest-first sort, so digraph categories map correctly (e.g. palatalization <code>[H]/[J]</code> where <code>[J]</code> holds <code>tʃ, dʒ</code>: declared order gives s→ʃ, t→tʃ, g→ɣ). The JS sorted members longest-first, silently mapping s→tʃ. Longest-first is still used for ''matching'', so multi-character phones still match greedily.
	* '''D15''' — <code>%</code> ("same as the target") substitutes on '''every''' occurrence in a result, so <code>%%</code> doubles the target (e.g. gemination <code>[C]/%%/_4</code>). The JS replaced only the first <code>%</code>. A literal per-cent sign in output is written <code>!%</code>.		* '''D15''' — <code>%</code> ("same as the target") substitutes on '''every''' occurrence in a result, so <code>%%</code> doubles the target (e.g. gemination <code>[C]/%%/_4</code>). The JS replaced only the first <code>%</code>. A literal per-cent sign in output is written <code>!%</code>.
			* '''D16''' (Auphen addition) — <code>~</code> (tilde) is a '''universal word boundary'''. The engine splits input on it, so each part is processed as its own word (its own <code>#</code> edges), and the parts are joined with a space; a <code>~</code> produced late by a rule becomes a plain display space. It does not clash with nasal vowels (precomposed like <code>ã</code>, or a combining U+0303 tilde — both distinct codepoints from ASCII <code>~</code>).

	Divergences from CWS (documented quirks fixed, not reproduced):		Divergences from CWS (documented quirks fixed, not reproduced):

Khurouan: Log the % fix (D15)

2026-07-18T12:35:39Z

Log the % fix (D15)

← Older revision		Revision as of 12:35, 18 July 2026
Line 15:		Line 15:
	* '''D6''' — local conditions inside an affix rule are cleanly false.		* '''D6''' — local conditions inside an affix rule are cleanly false.
	* '''D14''' — category→category mapping pairs members by '''declaration order''', not a longest-first sort, so digraph categories map correctly (e.g. palatalization <code>[H]/[J]</code> where <code>[J]</code> holds <code>tʃ, dʒ</code>: declared order gives s→ʃ, t→tʃ, g→ɣ). The JS sorted members longest-first, silently mapping s→tʃ. Longest-first is still used for ''matching'', so multi-character phones still match greedily.		* '''D14''' — category→category mapping pairs members by '''declaration order''', not a longest-first sort, so digraph categories map correctly (e.g. palatalization <code>[H]/[J]</code> where <code>[J]</code> holds <code>tʃ, dʒ</code>: declared order gives s→ʃ, t→tʃ, g→ɣ). The JS sorted members longest-first, silently mapping s→tʃ. Longest-first is still used for ''matching'', so multi-character phones still match greedily.
			* '''D15''' — <code>%</code> ("same as the target") substitutes on '''every''' occurrence in a result, so <code>%%</code> doubles the target (e.g. gemination <code>[C]/%%/_4</code>). The JS replaced only the first <code>%</code>. A literal per-cent sign in output is written <code>!%</code>.

	Divergences from CWS (documented quirks fixed, not reproduced):		Divergences from CWS (documented quirks fixed, not reproduced):

Khurouan: Document the engine's design and provenance

2026-07-18T00:58:13Z

Document the engine's design and provenance

New page

This page records the design, provenance, and behavioural divergences of [[Module:Auphen]] — out-of-frame notes, kept off the module source.

== Provenance ==

Auphen is a pure-Lua (Scribunto) port of a JavaScript sound-change engine, itself an implementation of ConWorkShop's '''PhoMo''' rule notation. Rule anatomy (target / change / environment / exception / else), the affix-vs-replace split, category sets, and the operator set follow PhoMo; the divergences below are deliberate. The engine is developed and tested off-wiki: a 334-case local suite covering every operation, ground-truth parity against the original JS engine, and the live YBS-PJ ruleset.

== Divergences ==

Fixed bugs in the source JS engine (each has a proving test off-wiki):
* '''D1''' — the ELS <code>@</code>-index is parsed and used as a fallback instance filter (the JS assigned it to the CHG index, so ELS indices were dead).
* '''D2''' — ditto <code>"</code> fires inside the exception as well as the environment (the JS dropped the "previous" flag).
* '''D3''' — <code>?</code>-reversal indices are computed on the clean word; negatives are not off by one, and positive <code>?n</code> starts at char ''n''.
* '''D4''' — positive-index affix insertion/deletion is not off by one.
* '''D5''' — circumfix <code>A#B</code> wraps the word (the JS concatenated A+B).
* '''D6''' — local conditions inside an affix rule are cleanly false.
* '''D14''' — category→category mapping pairs members by '''declaration order''', not a longest-first sort, so digraph categories map correctly (e.g. palatalization <code>[H]/[J]</code> where <code>[J]</code> holds <code>tʃ, dʒ</code>: declared order gives s→ʃ, t→tʃ, g→ɣ). The JS sorted members longest-first, silently mapping s→tʃ. Longest-first is still used for ''matching'', so multi-character phones still match greedily.

Divergences from CWS (documented quirks fixed, not reproduced):
* '''D7''' — a category/index resolving out of range skips the candidate (CWS deletes the target).
* '''D8''' — <code>!</code> escape makes the next character a literal.
* '''D9 / D10''' — movement <code>></code> and copy <code>>!</code> use clean semantics; CWS's off-by-one is not reproduced.
* '''D11''' — <code>-</code> word operations are opt-in (<code>opts.dashToSpace</code>).
* '''D12''' — the wildcard <code>*</code> in local conditions is defined and reliable.
* '''D13''' — a UTF-8 codepoint layer replaces the JS's UTF-16 indexing: combining marks are separate characters, and digraph phones work in categories (CWS bans them).

Kept behaviours: <code>@0</code>/<code>@-0</code> select nothing; an empty ELS is skip, not delete; only the first <code>%</code> in a result is substituted; capitals never survive to output; candidates are collected against the pre-rule word and applied together (no intra-rule bleeding).

== Performance ==

Category matching currently materialises the Cartesian product of the category references in a rule, so cost is (category size) to the power of (number of references) — dominated by large sets like <code>[C]</code>. A guard (<code>opts.explodeCap</code>, default 5,000,000) skips a pathological rule with a warning rather than exceeding Scribunto's CPU limit. '''Tracked follow-up:''' a streaming/positional matcher that tests membership per word-position, removing the ceiling entirely.