Module:Auphen

This is the documentation for Module:Auphen, the sound-change engine for the Yezur wiki. It applies an ordered ruleset of rewrite rules to a word and returns the result. It exports no #invoke entry point and is not called from wikitext: Module:Auphen/frame requires it, hands it the data for one language and backs Template:Auphen, so that {{auphen|word|code}} estimates a pronunciation and {{auphen|word|code|ruleset}} runs a named ruleset. The categories, the glyph-to-sound table and the rulesets themselves are pure data on the Module:Auphen/<code> pages; the engine holds no language-specific material. The rule notation is a port of PhoMo's; the behaviour log, the divergences from it and the module's provenance are kept on Module talk:Auphen.

Interface

Call	Purpose
Auphen.new(categories, ipa_rules, opts)	Builds an engine. categories maps each bracketed category name to an array of its members in declaration order; ipa_rules is the glyph-to-sound table as a string; opts holds the two settings below. Every argument may be omitted, and an engine built without ipa_rules does no glyph conversion.
eng:run(ruleset, input, toIpa)	Runs ruleset over input and returns the finished string. The input is split on spaces and on ~, each part is processed as its own word with its own word edges, and the parts are rejoined with spaces. With toIpa true the input is first lower-cased, sentence-final punctuation and the colon become |, other punctuation is dropped, and ipa_rules is applied.
eng:run_word(ruleset, word)	Runs the ruleset over a single word. Returns the finished word and a trace holding, for each rule, its text, the word after it and whether it applied.
eng.warnings	Warnings gathered while loading and running: an invalid category name, a category reference that cannot be resolved, an affix category used without an index, an affix index that matches nothing, a rule skipped for exceeding the cap below. Module:Auphen/frame does not read them.
opts.explodeCap	The ceiling on the number of variants one segment of a rule may expand to, 5,000,000 by default. A segment above it expands to nothing and the rule is skipped with a warning rather than left to exhaust the parser.
opts.dashToSpace	When true, hyphens in the finished word become spaces.

Neither option is reachable from wikitext: Module:Auphen/frame builds the engine without an opts table, so template calls run at the default cap and leave hyphens untouched.

Rule lines

A ruleset is plain text, one rule to a line; a line containing no / is ignored. Each rule has up to five fields, separated by /:

target / change / environment / exception / else

Field	Purpose
target	What the rule matches. # alone means the whole word; an empty target matches at every slot from before the first character to after the last.
change	What the match becomes. An empty field deletes it.
environment	Conditions on the surrounding word; the rule applies only where one of them holds. An empty field imposes no condition.
exception	Conditions written the same way; where one holds, the rule is blocked.
else	Applied in place of the change wherever the environment fails or an exception holds. An empty field leaves the match alone.

Rules run in order, each on the output of the last. The word is lower-cased before a rule sees it, and the finished word is re-cased to match the input: all lower, all upper, or otherwise capitalised. A rule whose target is # and whose change contains # or @ is treated as an affixation instead of a replacement; such a rule has no target position, so a condition containing _ never holds in one.

Notation

Notation	Meaning
[X]	A category: a bracketed capital followed by up to two more letters, its members listed on the language's data page. Members are tried longest first when matching but map by position, so the nth member of the target's category becomes the nth member of the change's.
[a,b,c]	A category written in place, members comma-separated, without being declared.
%	In the change, the text that was matched, so that %% doubles it; where the target is # it is the whole word. In a condition, replaced by the matched text before the condition is tested.
?	The matched text reversed, or the whole word where the target is #; it is read the same way in a condition. As the whole change, ?n reverses that text from position n, counting from the end where n is negative, and ?n^m reverses m characters from there.
_	The target's own slot, which makes the condition local: what stands left of _ must precede the match and what stands right of it must follow. A condition without _ is global and is tested against the whole word.
*	In a local condition, separates material that need not be adjacent: the segment nearest the slot must abut it, the others need only occur in order further out.
#	A word edge. The word is padded with # at each end, so a local condition tests an edge in place; in a global condition, material before the # must be word-initial and material after it word-final.
=	Counts occurrences, without overlap, in a global condition. [V]=3 holds at exactly three, [V]=>3 at three or more, [V]=<3 at three or fewer.
|	Separates alternative conditions; the field holds if any one of them does.
"	As a condition on its own, holds when the previous rule applied.
@n	Written at the end of the change, selects a single instance: the nth match of the target, counted from the end where n is negative. Instances are numbered by a left-to-right, longest-first scan, so overlapping members of one category count once. An index on the else field is used where the change carries none.
^m	After an index, the number of characters the change covers, counted from the start of the match rather than taken from the length of the target.
!x	Makes the operator character x an ordinary literal. The ! of the movement marker >! is not read as an escape.
~	A word boundary: each side is run as its own word and the output shows a space. A ~ produced by a rule becomes a plain space.

Affixation and movement

In an affixation rule the change is a template in which # stands for the word; positions are counted in characters from 1, and a negative position counts from the end. Apart from the movement forms, a template with no # in it leaves the word unchanged; an index and span with no material at all delete. Category references and ## are resolved before the template is applied, and % and ? are not read here.

Change	Effect
x#	Prefixes x; #x suffixes it and x#y circumfixes.
##	Appends a copy of the word, each further # appending another. With an index the copy is that slice of the word — ^m giving its length, one character by default — and is placed directly after it.
#x@n	Inserts x after character n; x#@n inserts it before.
x#@n^m	Replaces m characters from position n with x, or deletes them where the material is empty.
#[X]@n	The nth member of category X occurring in the word, used as affix material and here suffixed. The index is consumed by the category, so the template's # alone places the material.
>s@d^m	Moves m characters, one by default, from position s to position d, the destination being counted in the word with the block already removed. >! in place of > copies the block instead, counting the destination in the unchanged word.

Notes

ipa_rules is read as one glyph/sound pair per line, longest glyph first; anything it does not match passes through unchanged. It is applied only on the pronunciation path, where Module:Auphen/frame runs the data page's pronounce ruleset against its ipacats categories and wraps the result in slashes. A named ruleset is taken from sets and run against cats, over the orthography and with no glyph conversion. Either category set stands in for the other where a data page defines only one.

Worked examples for the languages with sound data are on Template:Auphen/testcases, which calls {{#invoke:Auphen/frame|raw}} for unformatted output. A missing word or code, a code with no data page and an unknown ruleset are reported rather than raised: {{auphen}} renders the message as an error and files the page in Category:Auphen errors, while raw returns it as a plain ERROR: line.

---

-- Module:Auphen -- phonological rule engine (pure Lua 5.1).
--
-- Applies an ordered ruleset of sound changes to a word: substitution,
-- deletion, affixation (prefix/suffix/circumfix), movement, reduplication,
-- metathesis, indexed and instance-based operations, and local/global
-- conditions, driven by category sets. Rule syntax, one per line:
--   target / change / environment / exception / else
-- Categories are written [X] (a bracketed name; members are declared in order,
-- and category-to-category mapping is positional by that order). '#' marks the
-- whole word (in target/result) or a word edge (in a local condition).
--
-- API:
--   local Auphen = require('Module:Auphen')
--   local eng = Auphen.new(categories, ipa_rules, opts)   -- opts.explodeCap
--   eng:run(ruleset, input, toIpa)   -- full pipeline -> string
--   eng:run_word(ruleset, word)      -- a single word  -> string
--   eng.warnings                     -- list of load/parse warning strings
--
-- Design notes, the full behaviour/divergence log, and provenance live on the
-- talk page: [[Module talk:Auphen]].

local Auphen = {}

-- ------------------------------------------------------------------
-- UTF-8 codepoint layer
-- ------------------------------------------------------------------

local schar = string.char
local sbyte = string.byte
local ssub  = string.sub
local sfind = string.find
local concat = table.concat

-- explode(s) -> array of codepoint substrings (1 element per codepoint)
local function explode(s)
  local out = {}
  local n = 0
  local i = 1
  local len = #s
  while i <= len do
    local b = sbyte(s, i)
    local w
    if b < 0x80 then w = 1
    elseif b < 0xE0 then w = 2
    elseif b < 0xF0 then w = 3
    else w = 4 end
    n = n + 1
    out[n] = ssub(s, i, i + w - 1)
    i = i + w
  end
  return out
end

-- encode a codepoint number to a UTF-8 byte string
local function cp_to_utf8(cp)
  if cp < 0x80 then
    return schar(cp)
  elseif cp < 0x800 then
    return schar(0xC0 + math.floor(cp / 0x40),
                 0x80 + (cp % 0x40))
  elseif cp < 0x10000 then
    return schar(0xE0 + math.floor(cp / 0x1000),
                 0x80 + (math.floor(cp / 0x40) % 0x40),
                 0x80 + (cp % 0x40))
  else
    return schar(0xF0 + math.floor(cp / 0x40000),
                 0x80 + (math.floor(cp / 0x1000) % 0x40),
                 0x80 + (math.floor(cp / 0x40) % 0x40),
                 0x80 + (cp % 0x40))
  end
end

-- ------------------------------------------------------------------
-- Case tables (ulower / uupper)
-- ------------------------------------------------------------------

local LOWER = {}   -- upper char -> lower char
local UPPER = {}   -- lower char -> upper char

local function add_pair(u, l)
  local us, ls = cp_to_utf8(u), cp_to_utf8(l)
  LOWER[us] = ls
  UPPER[ls] = us
end

-- ASCII A-Z
for c = 0x41, 0x5A do add_pair(c, c + 0x20) end
-- Latin-1 Supplement (skip 0xD7 multiplication sign; 0xDF sharp s has no pair)
for c = 0xC0, 0xDE do
  if c ~= 0xD7 then add_pair(c, c + 0x20) end
end
-- Latin Extended-A well-behaved runs
for c = 0x100, 0x137, 2 do add_pair(c, c + 1) end   -- even upper / odd lower
for c = 0x139, 0x148, 2 do add_pair(c, c + 1) end   -- odd upper / even lower
for c = 0x14A, 0x177, 2 do add_pair(c, c + 1) end   -- even upper / odd lower
add_pair(0x178, 0xFF)                               -- Y-diaeresis / y-diaeresis
for c = 0x179, 0x17E, 2 do add_pair(c, c + 1) end   -- odd upper / even lower
-- Greek Alpha-Omega (skip reserved 0x3A2)
for c = 0x391, 0x3A9 do
  if c ~= 0x3A2 then add_pair(c, c + 0x20) end
end
-- Final sigma: accept on input, upper-cases to Sigma
UPPER[cp_to_utf8(0x3C2)] = cp_to_utf8(0x3A3)

local function ulower(s)
  local a = explode(s)
  for i = 1, #a do a[i] = LOWER[a[i]] or a[i] end
  return concat(a)
end

local function uupper(s)
  local a = explode(s)
  for i = 1, #a do a[i] = UPPER[a[i]] or a[i] end
  return concat(a)
end

-- capitalize: first codepoint upper, rest lower
local function ucapitalize(s)
  local a = explode(s)
  if #a == 0 then return s end
  local out = {}
  out[1] = UPPER[a[1]] or a[1]
  for i = 2, #a do out[i] = LOWER[a[i]] or a[i] end
  return concat(out)
end

-- ------------------------------------------------------------------
-- Escape layer (`!x`). Operator chars map to Private-Use codepoints so
-- the rest of the pipeline treats them as ordinary literals; unescape at
-- the very end of run_word restores the real characters.
-- ------------------------------------------------------------------

local ESC = {}     -- operator char -> PUA box char
local UNESC = {}   -- PUA box char -> operator char
do
  local ops = { '[', ']', '{', '}', '(', ')', '@', '^', '#', '%', '?',
                '>', '<', '_', '=', '|', '*', '!', '-', ',', '.', '+' }
  for i = 1, #ops do
    local pua = cp_to_utf8(0xE000 + i)
    ESC[ops[i]] = pua
    UNESC[pua] = ops[i]
  end
end

-- turn `!x` into a boxed literal (operator -> PUA, else the bare char)
local function apply_escapes(seg)
  if not sfind(seg, '!', 1, true) then return seg end
  local a = explode(seg)
  local out = {}
  local n = 0
  local i = 1
  while i <= #a do
    if a[i] == '!' and n >= 1 and out[n] == '>' then
      -- the `!` of the movement copy marker `>!` is not an escape
      n = n + 1; out[n] = '!'; i = i + 1
    elseif a[i] == '!' and i < #a then
      local nx = a[i + 1]
      n = n + 1
      out[n] = ESC[nx] or nx
      i = i + 2
    else
      n = n + 1
      out[n] = a[i]
      i = i + 1
    end
  end
  return concat(out)
end

local function unescape(s)
  local a = explode(s)
  local changed = false
  for i = 1, #a do
    local u = UNESC[a[i]]
    if u then a[i] = u; changed = true end
  end
  if changed then return concat(a) end
  return s
end

-- ------------------------------------------------------------------
-- Small string helpers (ASCII-only separators: byte-safe on UTF-8)
-- ------------------------------------------------------------------

local function split_plain(s, sep)
  local out = {}
  local n = 0
  local start = 1
  while true do
    local i = sfind(s, sep, start, true)
    if not i then
      n = n + 1; out[n] = ssub(s, start)
      break
    end
    n = n + 1; out[n] = ssub(s, start, i - 1)
    start = i + #sep
  end
  return out
end

local function trim(s)
  return (s:gsub('^%s*(.-)%s*$', '%1'))
end

-- ------------------------------------------------------------------
-- Codepoint-array position helpers (0-based indexing)
-- ------------------------------------------------------------------

-- char at 0-based position p (nil if out of range)
local function jchar(W, p)
  if p < 0 then return nil end
  return W[p + 1]
end

-- substring [a, b) 0-based, joined to a string
local function jsub(W, a, b)
  if b == nil then b = #W end
  if a < 0 then a = 0 end
  local t = {}
  local k = 0
  for i = a + 1, b do k = k + 1; t[k] = W[i] end
  return concat(t)
end

-- first index (0-based) of needle array within W array at/after `from`
local function jindexof(W, needle, from)
  local hl, nl = #W, #needle
  if nl == 0 then return -1 end
  for i = (from or 0) + 1, hl - nl + 1 do
    local ok = true
    for k = 1, nl do
      if W[i + k - 1] ~= needle[k] then ok = false; break end
    end
    if ok then return i - 1 end
  end
  return -1
end

-- count non-overlapping occurrences of needle array in W array
local function jcount(W, needle)
  local nl = #needle
  if nl == 0 then return 0 end
  local c = 0
  local p = jindexof(W, needle, 0)
  while p ~= -1 do
    c = c + 1
    p = jindexof(W, needle, p + nl)
  end
  return c
end

-- ------------------------------------------------------------------
-- Engine
-- ------------------------------------------------------------------

local Engine = {}
Engine.__index = Engine

local function valid_cat_name(name)
  -- ^\[[A-Z][a-zA-Z]{0,2}\]$
  return sfind(name, '^%[%u[%a]?[%a]?%]$') ~= nil
end

function Engine:warn(msg)
  local w = self.warnings
  w[#w + 1] = msg
end

function Engine:set_cats(categories)
  local out = {}
  for name, members in pairs(categories) do
    if out[name] ~= nil then
      self:warn('Category ' .. name .. ' already exists and has been skipped.')
    elseif not valid_cat_name(name) then
      self:warn('Category name ' .. name .. ' is not valid and has been skipped.')
    else
      -- Store in DECLARATION order. Category->category mapping is positional
      -- (member i of TRG cat -> member i of CHG cat), so the stored index must
      -- follow the author's declared order. Longest-first is applied only where
      -- MATCHING needs it (scan_members; replace-path overlap resolution), never
      -- to the stored index -- so multi-char phones still match greedily while
      -- cat->cat maps stay aligned. See D14.
      local copy = {}
      for i = 1, #members do copy[i] = members[i] end
      out[name] = copy
    end
  end
  self.categories = out
end

function Auphen.new(categories, ipa_rules, opts)
  local self = setmetatable({}, Engine)
  self.warnings = {}
  self.opts = opts or {}
  self:set_cats(categories or {})
  self.ipa_rules = ipa_rules
  return self
end

-- explode_string(str): split into literal runs and bracket runs; each
-- registered/temporary category contributes its members with member
-- indices; cross-product -> list of { word=, indcs={} } alternatives.
function Engine:explode_string(str)
  local a = explode(str)
  local parts = {}       -- ordered list of {text=, isCat=bool}
  local buf = {}
  local i = 1
  local function flush()
    if #buf > 0 then parts[#parts + 1] = { text = concat(buf) }; buf = {} end
  end
  while i <= #a do
    if a[i] == '[' then
      local j = i + 1
      while j <= #a and a[j] ~= ']' do j = j + 1 end
      if j <= #a then
        flush()
        parts[#parts + 1] = { text = concat(a, '', i, j), isCat = true }
        i = j + 1
      else
        buf[#buf + 1] = a[i]; i = i + 1
      end
    else
      buf[#buf + 1] = a[i]; i = i + 1
    end
  end
  flush()

  -- build option lists
  local options = {}
  for _, part in ipairs(parts) do
    local cat = self.categories[part.text]
    if cat then
      local opt = {}
      for idx = 1, #cat do opt[idx] = { text = cat[idx], index = idx } end
      options[#options + 1] = opt
    elseif part.isCat and sfind(part.text, ',', 1, true) then
      -- temporary category [a,b,c]
      local pa = explode(part.text)
      local inner = concat(pa, '', 2, #pa - 1)
      local vals = split_plain(inner, ',')
      local opt = {}
      for idx = 1, #vals do opt[idx] = { text = trim(vals[idx]), index = idx } end
      options[#options + 1] = opt
    else
      -- literal run (unregistered bracket stays literal, brackets included)
      options[#options + 1] = { { text = part.text, index = nil } }
    end
  end

  -- Guard the Cartesian blow-up: product = (category size)^(#refs), dominated
  -- by large categories like [C]. Interim cap -- the streaming/positional
  -- matcher (tracked) will remove this ceiling. Over cap: warn + no-op (the
  -- rule matches nothing) rather than hang the page.
  local product = 1
  for _, opt in ipairs(options) do product = product * #opt end
  if product > (self.opts.explodeCap or 5000000) then
    self:warn('Rule too complex: category cross-product of ' .. product ..
              ' exceeds the cap; this rule was skipped. Segment: ' .. str)
    return {}
  end

  -- cross product
  local acc = { { word = '', indcs = {} } }
  for _, opt in ipairs(options) do
    local nxt = {}
    local m = 0
    for _, prev in ipairs(acc) do
      for _, o in ipairs(opt) do
        local indcs = {}
        for k = 1, #prev.indcs do indcs[k] = prev.indcs[k] end
        if o.index ~= nil then indcs[#indcs + 1] = o.index end
        m = m + 1
        nxt[m] = { word = prev.word .. o.text, indcs = indcs }
      end
    end
    acc = nxt
  end
  return acc
end

-- match the reversal template `?(-?\d*)(\^\d+)?` exactly; returns
-- idxStr, spanStr (either may be '' / nil) or nil if not a reversal.
local function match_reversal(s)
  local rest = s:match('^%?(.*)$')
  if not rest then return nil end
  local idx, span = rest:match('^(%-?%d*)%^(%d+)$')
  if idx then return idx, span end
  local idx2 = rest:match('^(%-?%d*)$')
  if idx2 ~= nil then return idx2, nil end
  return nil
end

-- reverse a codepoint span of `word` (a string) per clean-word semantics
local function reverse_span(word, idxStr, spanStr)
  local a = explode(word)
  local len = #a
  local start0 = 0            -- 0-based start
  if idxStr and idxStr ~= '' then
    local n = tonumber(idxStr)
    if n and n > 0 then
      start0 = n - 1
    elseif n and n < 0 then
      start0 = len + n
    end
  end
  local spanLen
  if spanStr and spanStr ~= '' then
    spanLen = tonumber(spanStr)
  else
    spanLen = len - start0
  end
  if start0 < 0 or start0 >= len then return word end
  if start0 + spanLen > len then spanLen = len - start0 end
  if spanLen <= 0 then return word end
  local out = {}
  for i = 1, start0 do out[i] = a[i] end
  -- reversed middle
  for k = 0, spanLen - 1 do
    out[start0 + 1 + k] = a[start0 + spanLen - k]
  end
  for i = start0 + spanLen + 1, len do out[i] = a[i] end
  return concat(out)
end

-- replace first plain occurrence of `needle` in `s` with `repl`
local function replace_first(s, needle, repl)
  local i = sfind(s, needle, 1, true)
  if not i then return s end
  return ssub(s, 1, i - 1) .. repl .. ssub(s, i + #needle)
end

-- resolve_replacement_string(template, trgObj, mode, revWord)
-- Returns the concrete replacement string, or nil to skip the candidate.
function Engine:resolve_replacement_string(template, trgObj, mode, revWord)
  if template == '' and mode == 'els' then return nil end
  local copy = template
  local i = 1
  -- resolve [cat] refs left to right, i-th bracket <-> i-th TRG index
  while true do
    local s, e = sfind(copy, '%[.-%]')
    if not s then break end
    local catName = ssub(copy, s, e)
    local catVals = self.categories[catName]
    if not catVals and sfind(catName, ',', 1, true) then
      local pa = explode(catName)
      local inner = concat(pa, '', 2, #pa - 1)
      local raw = split_plain(inner, ',')
      catVals = {}
      for k = 1, #raw do catVals[k] = trim(raw[k]) end
    end
    local trgIndex = trgObj.indcs and trgObj.indcs[i] or nil
    if catVals and trgIndex ~= nil and catVals[trgIndex] ~= nil then
      copy = ssub(copy, 1, s - 1) .. catVals[trgIndex] .. ssub(copy, e + 1)
    else
      self:warn('Category error resolving ' .. template ..
                ' for ' .. (trgObj.word or ''))
      return nil
    end
    i = i + 1
  end

  local rw = revWord or trgObj.word
  if rw and rw ~= '' then
    local idxStr, spanStr = match_reversal(copy)
    if idxStr ~= nil then
      copy = reverse_span(rw, idxStr, spanStr)
    else
      -- fallback: first literal `?` -> fully reversed word
      if sfind(copy, '?', 1, true) then
        local ra = explode(rw)
        local rev = {}
        for k = 1, #ra do rev[k] = ra[#ra - k + 1] end
        copy = replace_first(copy, '?', concat(rev))
      end
    end
    -- every `%` -> the word ("%" = same as TRG, so "%%" doubles it, e.g.
    -- gemination [C]/%%/_4). A literal per-cent in output is written "!%".
    if sfind(copy, '%', 1, true) then
      copy = concat(split_plain(copy, '%'), rw)
    end
  end
  return copy
end

-- ------------------------------------------------------------------
-- Condition helpers
-- ------------------------------------------------------------------

local function repl_all(s, a, b)
  if not sfind(s, a, 1, true) then return s end
  local out = {}
  local n = 0
  local start = 1
  while true do
    local i = sfind(s, a, start, true)
    if not i then n = n + 1; out[n] = ssub(s, start); break end
    n = n + 1; out[n] = ssub(s, start, i - 1)
    n = n + 1; out[n] = b
    start = i + #a
  end
  return concat(out)
end

local function str_reverse_cp(s)
  local a = explode(s)
  local r = {}
  for k = 1, #a do r[k] = a[#a - k + 1] end
  return concat(r)
end

local function split_arrays(s, sep)
  local raw = split_plain(s, sep)
  local out = {}
  for i = 1, #raw do out[i] = explode(unescape(raw[i])) end
  return out
end

-- region (an array) ends with the last segment (adjacent), earlier
-- segments occurring in order before it.
local function region_ends_with(textArr, segs)
  local N = #segs
  local last = segs[N]
  local tl = #textArr
  local ll = #last
  if ll > tl then return false end
  for k = 1, ll do
    if textArr[tl - ll + k] ~= last[k] then return false end
  end
  local limit = tl - ll        -- earlier segs must fit in [0, limit)
  local cursor = 0             -- 0-based
  for s = 1, N - 1 do
    local seg = segs[s]
    local sl = #seg
    local found = nil
    for start = cursor, limit - sl do
      local ok = true
      for k = 1, sl do
        if textArr[start + k] ~= seg[k] then ok = false; break end
      end
      if ok then found = start; break end
    end
    if not found then return false end
    cursor = found + sl
  end
  return true
end

-- region (an array) starts with the first segment (adjacent), the rest
-- occurring in order after it.
local function region_starts_with(textArr, segs)
  local first = segs[1]
  local fl = #first
  if fl > #textArr then return false end
  for k = 1, fl do
    if textArr[k] ~= first[k] then return false end
  end
  local cursor = fl            -- 0-based
  for s = 2, #segs do
    local seg = segs[s]
    local sl = #seg
    local found = nil
    for start = cursor, #textArr - sl do
      local ok = true
      for k = 1, sl do
        if textArr[start + k] ~= seg[k] then ok = false; break end
      end
      if ok then found = start; break end
    end
    if not found then return false end
    cursor = found + sl
  end
  return true
end


-- ------------------------------------------------------------------
-- Positional context matching
-- ------------------------------------------------------------------
-- A local environment is matched by walking the pattern against the word and
-- testing category membership in place. Enumerating the categories instead
-- (as explode_string does) is a Cartesian blow-up: an environment such as
-- [C]_[C][C][V] is a quarter of a million variants, built afresh at every
-- match position, which exhausts the interpreter before it can answer.

-- parse a context pattern into atoms: { lit = <codepoint> } | { cat = <array> }
function Engine:ctx_atoms(pat)
  local a = explode(pat)
  local atoms, i = {}, 1
  while i <= #a do
    if a[i] == '[' then
      local j = i + 1
      while j <= #a and a[j] ~= ']' do j = j + 1 end
      if j <= #a then
        local name = concat(a, '', i, j)
        local cat = self.categories[name]
        if not cat and sfind(name, ',', 1, true) then
          local pa = explode(name)
          local vals = split_plain(concat(pa, '', 2, #pa - 1), ',')
          cat = {}
          for k = 1, #vals do cat[k] = trim(vals[k]) end
        end
        if cat then
          atoms[#atoms + 1] = { cat = cat }
          i = j + 1
        else
          atoms[#atoms + 1] = { lit = a[i] }; i = i + 1
        end
      else
        atoms[#atoms + 1] = { lit = a[i] }; i = i + 1
      end
    else
      atoms[#atoms + 1] = { lit = a[i] }; i = i + 1
    end
  end
  return atoms
end

-- Walk `atoms` (from index k) against the word from 0-based position p. Members
-- of a category are tried longest-first, so multi-codepoint phones still match
-- greedily -- but the walk backtracks: where the longest member would swallow
-- material a later atom needs, or the target itself, a shorter member is tried
-- instead. `stop`, when given, requires the walk to end exactly there, which is
-- what a left context needs. Returns the end position, or nil for no match.
function Engine:ctx_walk(W, p, atoms, k, stop)
  if k > #atoms then
    if stop ~= nil and p ~= stop then return nil end
    return p
  end
  local at = atoms[k]
  if at.lit ~= nil then
    if jchar(W, p) ~= at.lit then return nil end
    return self:ctx_walk(W, p + 1, atoms, k + 1, stop)
  end
  -- the lengths of every member matching here, longest first, no duplicates
  local lens, seen = {}, {}
  for _, m in ipairs(at.cat) do
    local ma = explode(m)
    local ok = #ma > 0
    for i = 1, #ma do
      if jchar(W, p + i - 1) ~= ma[i] then ok = false; break end
    end
    if ok and not seen[#ma] then
      seen[#ma] = true
      lens[#lens + 1] = #ma
    end
  end
  table.sort(lens, function(a, b) return a > b end)
  for _, l in ipairs(lens) do
    local r = self:ctx_walk(W, p + l, atoms, k + 1, stop)
    if r ~= nil then return r end
  end
  return nil
end

-- match atoms forwards from 0-based position p; return the end position or nil
function Engine:ctx_match_fwd(W, p, atoms)
  return self:ctx_walk(W, p, atoms, 1, nil)
end

-- match atoms so that they end exactly at 0-based position pos
function Engine:ctx_match_back(W, pos, atoms)
  for start = pos, 0, -1 do
    if self:ctx_walk(W, start, atoms, 1, pos) ~= nil then return start end
  end
  return nil
end

function Engine:check_env(W, pos, trg, ENVs, prev)
  if #ENVs == 0 then return true end
  local trgLen = #explode(trg)
  local env_passed = false
  for _, rawENV in ipairs(ENVs) do
    if rawENV == '"' and prev then env_passed = true end
    local myENV = repl_all(rawENV, '%', trg)
    myENV = repl_all(myENV, '?', str_reverse_cp(trg))
    if sfind(myENV, '_', 1, true) then
      -- LOCAL condition (FALSE in affix rules, where pos is nil)
      if pos ~= nil and not sfind(myENV, '*', 1, true) then
        -- positional match: no category enumeration, so no blow-up
        local parts = split_plain(myENV, '_')
        local left = parts[1] or ''
        local right = parts[2] or ''
        local ok = true
        if left ~= '' then
          if self:ctx_match_back(W, pos, self:ctx_atoms(unescape(left))) == nil then
            ok = false
          end
        end
        if ok and right ~= '' then
          if self:ctx_match_fwd(W, pos + trgLen, self:ctx_atoms(unescape(right))) == nil then
            ok = false
          end
        end
        if ok then env_passed = true end
      elseif pos ~= nil then
        local variants = self:explode_string(myENV)
        for _, v in ipairs(variants) do
          local parts = split_plain(v.word, '_')
          local left = parts[1] or ''
          local right = parts[2] or ''
          local ok = true
          if left ~= '' then
            if sfind(left, '*', 1, true) then
              local textArr = {}
              for k = 1, pos do textArr[k] = W[k] end
              if not region_ends_with(textArr, split_arrays(left, '*')) then
                ok = false
              end
            else
              local la = explode(unescape(left))
              local ll = #la
              for k = 1, ll do
                if la[k] ~= jchar(W, pos - ll + (k - 1)) then ok = false; break end
              end
            end
          end
          if ok and right ~= '' then
            if sfind(right, '*', 1, true) then
              local textArr = {}
              local idx = 0
              for k = pos + trgLen + 1, #W do idx = idx + 1; textArr[idx] = W[k] end
              if not region_starts_with(textArr, split_arrays(right, '*')) then
                ok = false
              end
            else
              local ra = explode(unescape(right))
              local rl = #ra
              for k = 1, rl do
                if ra[k] ~= jchar(W, pos + (k - 1) + trgLen) then ok = false; break end
              end
            end
          end
          if ok then env_passed = true end
        end
      end
    else
      -- GLOBAL condition
      if sfind(myENV, '#', 1, true) then
        local variants = self:explode_string(myENV)
        for _, v in ipairs(variants) do
          local cp = true
          local hp = sfind(v.word, '#', 1, true)
          local wordstart = ssub(v.word, 1, hp - 1)
          local wordend = ssub(v.word, hp + 1)
          if wordstart ~= '' then
            local ns = explode(unescape(wordstart))
            local p = jindexof(W, ns, 0)
            if p == -1 or jsub(W, 0, p) ~= '#' then cp = false end
          end
          if wordend ~= '' then
            local ne = explode(unescape(wordend))
            local last = -1
            local q = jindexof(W, ne, 0)
            while q ~= -1 do last = q; q = jindexof(W, ne, q + 1) end
            if last == -1 or jsub(W, last + #ne) ~= '#' then cp = false end
          end
          if cp then env_passed = true end
        end
      elseif sfind(myENV, '=', 1, true) then
        local eparts = split_plain(myENV, '=')
        local countme = eparts[1]
        local checkstr = eparts[2] or ''
        local ctype = ssub(checkstr, 1, 1)
        if ctype == '>' or ctype == '<' then checkstr = ssub(checkstr, 2) end
        local target = tonumber(checkstr) or 0
        local cnt = 0
        local variants = self:explode_string(countme)
        for _, v in ipairs(variants) do
          cnt = cnt + jcount(W, explode(unescape(v.word)))
        end
        local cp = true
        if ctype == '<' then
          if cnt > target then cp = false end
        elseif ctype == '>' then
          if cnt < target then cp = false end
        else
          if cnt ~= target then cp = false end
        end
        if cp then env_passed = true end
      else
        local variants = self:explode_string(myENV)
        for _, v in ipairs(variants) do
          if jindexof(W, explode(unescape(v.word)), 0) ~= -1 then env_passed = true end
        end
      end
    end
  end
  return env_passed
end

function Engine:check_exc(W, pos, trg, EXCs, prev)
  if #EXCs == 0 then return true end
  return not self:check_env(W, pos, trg, EXCs, prev)
end

-- ------------------------------------------------------------------
-- Replace path
-- ------------------------------------------------------------------

local function parse_at(seg)
  -- returns strippedSeg, IND, SPA (for the replace path)
  local IND, SPA
  if sfind(seg, '@', 1, true) then
    local before, after = seg:match('^(.-)@(.*)$')
    seg = before
    if sfind(after, '^', 1, true) then
      local a2, b2 = after:match('^(.-)%^(.*)$')
      IND = tonumber(a2); SPA = tonumber(b2)
    else
      IND = tonumber(after)
    end
  end
  return seg, IND, SPA
end

function Engine:handle_rule(segs, word, prev)
  local rawTRG = segs[1] or ''
  local rawCHG = segs[2] or ''
  local ENV = segs[3] or ''
  local EXC = segs[4] or ''
  local ELS = segs[5] or ''

  local CHG, CHG_IND, CHG_SPA = parse_at(rawCHG)
  local ELS_stripped, ELS_IND, ELS_SPA = parse_at(ELS)
  ELS = ELS_stripped

  local cleanWord = word
  local padded = '#' .. word .. '#'
  local wholeWord = false
  local TRG = rawTRG
  if rawTRG == '#' then
    TRG = padded
    wholeWord = true
  end
  local W = explode(padded)

  local ENVs = ENV ~= '' and split_plain(ENV, '|') or {}
  local EXCs = EXC ~= '' and split_plain(EXC, '|') or {}

  local TRGs = self:explode_string(TRG)
  local candidates = {}
  local success = false

  -- Collect the matches of every expanded target before filtering by index. An
  -- @-index numbers instances of the target as a whole, so on a category target
  -- `[V]/%3@-1` is "the last vowel in the word" -- not "the last of every vowel
  -- letter", which is what per-member numbering would mark.
  local hits = {}
  for order, myTRG in ipairs(TRGs) do
    local trgReal = unescape(myTRG.word)
    local needle = explode(trgReal)
    local revWord = wholeWord and cleanWord or trgReal
    local resolved_CHG = self:resolve_replacement_string(CHG, myTRG, 'chg', revWord)
    if resolved_CHG ~= nil then
      local resolved_ELS = nil
      if ELS ~= '' then
        resolved_ELS = self:resolve_replacement_string(ELS, myTRG, 'els', revWord)
      end
      local function hit(p)
        hits[#hits + 1] = { pos = p, len = #needle, order = order, trg = trgReal,
                            chg = resolved_CHG, els = resolved_ELS }
      end
      if #needle == 0 then
        for p = 1, #W - 1 do hit(p) end
      else
        local p = jindexof(W, needle, 0)
        while p ~= -1 do
          hit(p)
          p = jindexof(W, needle, p + 1)
        end
      end
    end
  end

  -- reading order: by position, longest member first, then declaration order
  table.sort(hits, function(a, b)
    if a.pos ~= b.pos then return a.pos < b.pos end
    if a.len ~= b.len then return a.len > b.len end
    return a.order < b.order
  end)

  -- instance filter
  local active = CHG_IND
  if active == nil then active = ELS_IND end
  if active ~= nil then
    -- Number the instances by a left-to-right, longest-first scan, so that
    -- overlapping members of one category (`[V]` holding both `ou` and `u`)
    -- count as one instance -- the same overlap rule the candidate merge below
    -- applies to the results.
    local seq, last = {}, -1
    for _, h in ipairs(hits) do
      if h.pos >= last then seq[#seq + 1] = h; last = h.pos + h.len end
    end
    local t0
    if active > 0 then t0 = active - 1
    elseif active < 0 then t0 = #seq + active
    else t0 = -1 end
    if t0 >= 0 and seq[t0 + 1] ~= nil then
      hits = { seq[t0 + 1] }
    else
      hits = {}
    end
  end

  for _, h in ipairs(hits) do
    local passed_ENV = self:check_env(W, h.pos, h.trg, ENVs, prev)
    local passed_EXC = self:check_exc(W, h.pos, h.trg, EXCs, prev)
    local span_len = h.len
    if passed_ENV and passed_EXC then
      if CHG_SPA ~= nil then span_len = CHG_SPA end
      success = true
      candidates[#candidates + 1] =
        { start = h.pos, endp = h.pos + span_len, rep = h.chg, length = span_len }
    elseif h.els ~= nil then
      if ELS_SPA ~= nil then span_len = ELS_SPA
      elseif CHG_SPA ~= nil then span_len = CHG_SPA end
      candidates[#candidates + 1] =
        { start = h.pos, endp = h.pos + span_len, rep = h.els, length = span_len }
    end
  end

  table.sort(candidates, function(a, b)
    if a.start ~= b.start then return a.start < b.start end
    return a.length > b.length
  end)
  local finals = {}
  local last = -1
  for _, c in ipairs(candidates) do
    if c.start >= last then finals[#finals + 1] = c; last = c.endp end
  end
  for k = #finals, 1, -1 do
    local c = finals[k]
    local before = jsub(W, 0, c.start)
    local after = jsub(W, c.endp)
    W = explode(before .. c.rep .. after)
  end
  local outp = repl_all(concat(W), '#', '')
  return outp, success
end

-- ------------------------------------------------------------------
-- Affix path
-- ------------------------------------------------------------------

-- scan clean word for occurrences of any member (longest-first), return
-- the list of matched member strings in order.
function Engine:scan_members(clean, members)
  local ms = {}
  for i = 1, #members do ms[i] = members[i] end
  -- stable sort descending by codepoint length
  local lens = {}
  for i = 1, #ms do lens[i] = #explode(ms[i]) end
  for i = 2, #ms do
    local v, lv = ms[i], lens[i]
    local j = i - 1
    while j >= 1 and lens[j] < lv do
      ms[j + 1] = ms[j]; lens[j + 1] = lens[j]; j = j - 1
    end
    ms[j + 1] = v; lens[j + 1] = lv
  end
  local A = explode(clean)
  local L = #A
  local found = {}
  local i = 1
  while i <= L do
    local matched = nil
    for _, m in ipairs(ms) do
      local ma = explode(m)
      local ml = #ma
      if ml > 0 and i + ml - 1 <= L then
        local ok = true
        for k = 1, ml do
          if A[i + k - 1] ~= ma[k] then ok = false; break end
        end
        if ok then matched = m; i = i + ml; break end
      end
    end
    if matched then
      found[#found + 1] = matched
    else
      i = i + 1
    end
  end
  return found
end

-- parse an affix template's @index, honoring the '#'-salvage.
local function parse_affix_at(seg)
  local IND, SPA
  if sfind(seg, '@', 1, true) then
    local before, after = seg:match('^(.-)@(.*)$')
    seg = before
    if sfind(after, '#', 1, true) then
      if ssub(after, -1) == '#' then seg = seg .. '#' end
      after = repl_all(after, '#', '')
    end
    if sfind(after, '^', 1, true) then
      local a2, b2 = after:match('^(.-)%^(.*)$')
      IND = tonumber(a2); SPA = tonumber(b2)
    else
      IND = tonumber(after)
    end
  end
  return seg, IND, SPA
end

-- resolve category refs (consuming the index) and expand `##` reduplication.
function Engine:affix_prep(tpl, ind, span, clean)
  if sfind(tpl, '[', 1, true) then
    while true do
      local s, e = sfind(tpl, '%[.-%]')
      if not s then break end
      local catName = ssub(tpl, s, e)
      local catVals = self.categories[catName]
      if not catVals and sfind(catName, ',', 1, true) then
        local pa = explode(catName)
        local inner = concat(pa, '', 2, #pa - 1)
        local raw = split_plain(inner, ',')
        catVals = {}
        for k = 1, #raw do catVals[k] = trim(raw[k]) end
      end
      local replacement = ''
      if ind ~= nil and catVals then
        local found = self:scan_members(clean, catVals)
        local t0
        if ind > 0 then t0 = ind - 1
        elseif ind < 0 then t0 = #found + ind
        else t0 = -1 end
        if t0 >= 0 and found[t0 + 1] then
          replacement = found[t0 + 1]
        else
          self:warn('Affix index @' .. tostring(ind) ..
                    ' for ' .. catName .. ' not found in ' .. clean)
        end
        ind = nil
      elseif catVals then
        self:warn('Affix category ' .. catName ..
                  ' without index resolves to empty in ' .. clean)
      end
      tpl = ssub(tpl, 1, s - 1) .. replacement .. ssub(tpl, e + 1)
    end
  end
  if sfind(tpl, '##', 1, true) then
    local segment = clean
    if ind ~= nil then
      local A = explode(clean)
      local L = #A
      local start0 = (ind > 0) and (ind - 1) or (L + ind)
      local len = (span ~= nil) and span or 1
      local s1 = start0; if s1 < 0 then s1 = 0 end
      local e1 = start0 + len; if e1 > L then e1 = L end
      local seg = {}
      for k = s1 + 1, e1 do seg[#seg + 1] = A[k] end
      segment = concat(seg)
      ind = start0 + len
      span = 0
    end
    tpl = tpl:gsub('#+', function(m)
      local copies = #m - 1
      if copies < 1 then return m end
      return '#' .. string.rep(segment, copies)
    end)
  end
  return tpl, ind, span
end

function Engine:apply_affix_logic(clean, template, index, span)
  local A = explode(clean)
  local L = #A
  local material = repl_all(template, '#', '')
  local tarr = explode(template)
  local startsHash = (tarr[1] == '#')

  if index ~= nil then
    local p
    if index > 0 then p = index
    elseif index < 0 then p = L + index + 1
    else p = 0 end
    local sp = span or 0
    local ma = explode(material)
    if sp > 0 then
      local delStart = p; if delStart < 1 then delStart = 1 end
      local delEnd = delStart + sp - 1; if delEnd > L then delEnd = L end
      local out = {}
      for k = 1, delStart - 1 do out[#out + 1] = A[k] end
      for k = 1, #ma do out[#out + 1] = ma[k] end
      for k = delEnd + 1, L do out[#out + 1] = A[k] end
      return concat(out)
    else
      local gap
      if startsHash then gap = p else gap = p - 1 end
      if gap < 0 then gap = 0 end
      if gap > L then gap = L end
      local out = {}
      for k = 1, gap do out[#out + 1] = A[k] end
      for k = 1, #ma do out[#out + 1] = ma[k] end
      for k = gap + 1, L do out[#out + 1] = A[k] end
      return concat(out)
    end
  else
    -- prefix / suffix / circumfix, split at the first real '#'
    local hp = sfind(template, '#', 1, true)
    if not hp then return clean .. material end
    local left = ssub(template, 1, hp - 1)
    local right = ssub(template, hp + 1)
    return left .. clean .. right
  end
end

-- movement / copy (gap-fill). copyMode from '>!'.
local function do_move(clean, copyMode, src, dest, span)
  local A = explode(clean)
  local L = #A
  local s1
  if src > 0 then s1 = src elseif src < 0 then s1 = L + src + 1 else s1 = 1 end
  if s1 < 1 then s1 = 1 end
  if s1 > L then s1 = L + 1 end
  local sl = span or 1
  if sl < 1 then sl = 1 end
  local e1 = s1 + sl - 1
  if e1 > L then e1 = L end
  local block = {}
  for k = s1, e1 do block[#block + 1] = A[k] end
  if copyMode then
    local d = dest
    local dpos
    if d > 0 then dpos = d elseif d < 0 then dpos = L + d + 1 else dpos = 1 end
    if dpos < 1 then dpos = 1 end
    if dpos > L + 1 then dpos = L + 1 end
    local out = {}
    for k = 1, dpos - 1 do out[#out + 1] = A[k] end
    for k = 1, #block do out[#out + 1] = block[k] end
    for k = dpos, L do out[#out + 1] = A[k] end
    return concat(out)
  else
    local reduced = {}
    for k = 1, L do
      if k < s1 or k > e1 then reduced[#reduced + 1] = A[k] end
    end
    local rl = #reduced
    local d = dest
    local dpos
    if d > 0 then dpos = d elseif d < 0 then dpos = rl + d + 1 else dpos = 1 end
    if dpos < 1 then dpos = 1 end
    if dpos > rl + 1 then dpos = rl + 1 end
    local out = {}
    for k = 1, dpos - 1 do out[#out + 1] = reduced[k] end
    for k = 1, #block do out[#out + 1] = block[k] end
    for k = dpos, rl do out[#out + 1] = reduced[k] end
    return concat(out)
  end
end

function Engine:handle_affix(segs, word, prev)
  local TRG = segs[1] or ''
  local CHG = segs[2] or ''
  local ENV = segs[3] or ''
  local EXC = segs[4] or ''
  local ELS = segs[5] or ''
  local clean = word
  local W = explode('#' .. word .. '#')
  local ENVs = ENV ~= '' and split_plain(ENV, '|') or {}
  local EXCs = EXC ~= '' and split_plain(EXC, '|') or {}

  -- movement / copy path (real leading '>')
  local chgArr = explode(CHG)
  if chgArr[1] == '>' then
    local rest = ssub(CHG, 2)
    local copyMode = false
    if ssub(rest, 1, 1) == '!' then copyMode = true; rest = ssub(rest, 2) end
    local srcStr, destPart = rest:match('^(.-)@(.*)$')
    if srcStr then
      local destStr, spanStr
      if sfind(destPart, '^', 1, true) then
        destStr, spanStr = destPart:match('^(.-)%^(.*)$')
      else
        destStr = destPart
      end
      local src = tonumber(srcStr)
      local dest = tonumber(destStr)
      local span = spanStr and tonumber(spanStr) or 1
      local passed_ENV = self:check_env(W, nil, TRG, ENVs, prev)
      local passed_EXC = self:check_exc(W, nil, TRG, EXCs, prev)
      if src and dest and passed_ENV and passed_EXC then
        return do_move(clean, copyMode, src, dest, span), true
      end
      return clean, false
    end
  end

  local CHG_stripped, CHG_IND, CHG_SPA = parse_affix_at(CHG)
  CHG = CHG_stripped
  local ELS_IND, ELS_SPA
  ELS, ELS_IND, ELS_SPA = parse_affix_at(ELS)

  CHG, CHG_IND, CHG_SPA = self:affix_prep(CHG, CHG_IND, CHG_SPA, clean)
  if ELS ~= '' then
    ELS, ELS_IND, ELS_SPA = self:affix_prep(ELS, ELS_IND, ELS_SPA, clean)
  end

  local passed_ENV = self:check_env(W, nil, TRG, ENVs, prev)
  local passed_EXC = self:check_exc(W, nil, TRG, EXCs, prev)
  local success = false
  local result = clean
  if passed_ENV and passed_EXC then
    success = true
    if sfind(CHG, '#', 1, true) then
      result = self:apply_affix_logic(clean, CHG, CHG_IND, CHG_SPA)
    elseif CHG == '' and (CHG_IND ~= nil or CHG_SPA ~= nil) then
      result = self:apply_affix_logic(clean, CHG, CHG_IND, CHG_SPA)
    end
  else
    if ELS ~= '' then
      if sfind(ELS, '#', 1, true) then
        result = self:apply_affix_logic(clean, ELS, ELS_IND, ELS_SPA)
      elseif ELS == '' and (ELS_IND ~= nil or ELS_SPA ~= nil) then
        result = self:apply_affix_logic(clean, ELS, ELS_IND, ELS_SPA)
      end
    end
  end
  return result, success
end

-- ------------------------------------------------------------------
-- Pipeline
-- ------------------------------------------------------------------

function Engine:apply_rule(segs, word, prev)
  local lower = ulower(word)
  local rawTRG = segs[1] or ''
  local rawCHG = segs[2] or ''
  if rawTRG == '#' and (sfind(rawCHG, '#', 1, true) or sfind(rawCHG, '@', 1, true)) then
    return self:handle_affix(segs, lower, prev)
  else
    return self:handle_rule(segs, lower, prev)
  end
end

function Engine:convert_to_ipa(input)
  if not self.ipa_rules then return input end
  local map = {}
  local keys = {}
  local lines = split_plain(self.ipa_rules, '\n')
  for _, line in ipairs(lines) do
    line = trim(line)
    if line ~= '' then
      local parts = split_plain(line, '/')
      if #parts >= 2 then
        local key = trim(parts[1])
        local val = trim(parts[2])
        if map[key] == nil then keys[#keys + 1] = key end
        map[key] = val
      end
    end
  end
  if #keys == 0 then return input end
  table.sort(keys, function(a, b) return #explode(a) > #explode(b) end)
  local A = explode(input)
  local L = #A
  local out = {}
  local i = 1
  while i <= L do
    local matched = nil
    for _, k in ipairs(keys) do
      local ka = explode(k)
      local kl = #ka
      if kl > 0 and i + kl - 1 <= L then
        local ok = true
        for t = 1, kl do
          if A[i + t - 1] ~= ka[t] then ok = false; break end
        end
        if ok then matched = k; break end
      end
    end
    if matched then
      out[#out + 1] = map[matched]
      i = i + #explode(matched)
    else
      out[#out + 1] = A[i]
      i = i + 1
    end
  end
  return concat(out)
end

function Engine:run_word(ruleset, word)
  word = word or ''
  ruleset = ruleset or ''
  local capv
  if word == ulower(word) then capv = 'lower'
  elseif word == uupper(word) then capv = 'upper'
  else capv = 'capitalized' end

  local lines = split_plain(ruleset, '\n')
  local prev = false
  local trace = {}
  for _, line in ipairs(lines) do
    line = trim(line)
    if sfind(line, '/', 1, true) then
      local segs = split_plain(line, '/')
      for i = 1, #segs do segs[i] = apply_escapes(segs[i]) end
      local out, ok = self:apply_rule(segs, word, prev)
      word = out
      prev = ok and true or false
      trace[#trace + 1] = { rule = line, output = word, success = prev }
    end
  end

  if capv == 'lower' then word = ulower(word)
  elseif capv == 'upper' then word = uupper(word)
  else word = ucapitalize(word) end

  if self.opts.dashToSpace then word = repl_all(word, '-', ' ') end
  word = unescape(word)
  return word, trace
end

function Engine:run(ruleset, input, ipa)
  input = input or ''
  if ipa then
    input = ulower(input)
    input = repl_all(input, '. ', ' | ')
    input = repl_all(input, '.', '')
    input = repl_all(input, '? ', ' | ')
    input = repl_all(input, '?', '')
    input = repl_all(input, '! ', ' | ')
    input = repl_all(input, '!', '')
    input = repl_all(input, '" ', '')
    input = repl_all(input, ' "', '')
    input = repl_all(input, " '", '')
    input = repl_all(input, "' ", '')
    input = repl_all(input, ',', '')
    input = repl_all(input, ';', '')
    input = repl_all(input, ':', ' | ')
    input = self:convert_to_ipa(input)
  end
  -- `~` is a universal word boundary: split on it so each part is processed as
  -- its own word (its own `#` edges), then join the parts with a space. A `~`
  -- produced late by a rule becomes a plain display space.
  input = repl_all(input, '~', ' ')
  local words = split_plain(input, ' ')
  local out = {}
  for i = 1, #words do
    out[i] = repl_all(self:run_word(ruleset, words[i]), '~', ' ')
  end
  return concat(out, ' ')
end

return Auphen