Skip to content

fix(tts/kokoro-ane): stem possessives instead of G2P-ing the whole token - #884

Open
akmittal006 wants to merge 1 commit into
FluidInference:mainfrom
akmittal006:fix/kokoro-ane-possessive-clitic
Open

fix(tts/kokoro-ane): stem possessives instead of G2P-ing the whole token#884
akmittal006 wants to merge 1 commit into
FluidInference:mainfrom
akmittal006:fix/kokoro-ane-possessive-clitic

Conversation

@akmittal006

Copy link
Copy Markdown

The Misaki lexicon stores the -s clitic as its own 's entry and carries no
glued possessive keys, so today's, someone's and the boss's miss every
lexicon tier in KokoroAneEnglishPhonemizer.resolveWord and the whole inflected
token reaches the BART G2P fallback, which mangles it.

Rendered and transcribed with whisper.cpp ggml-small.en:

Input Before After
Someone's coat is here. "Samian's coat is here" "Someone's coat is here"
Today's plan is simple. mangled stem "Today's plan is simple"
the boss's office mangled stem "the boss's office"
the cat's bowl mangled stem "the cat's bowl"

Python Misaki does not have this problem because Lexicon.__call__ falls
through to stem_s: the stem is looked up on its own and the clitic phoneme is
appended by rule. This ports that rule.

The rule

After a full lexicon miss (and after the #710 initialism spell-out), a token
ending in 's — case-insensitively, and after the #774 apostrophe folding, so
Today's, TODAY'S and Today’s all qualify — resolves its stem through the
normal chain and appends the clitic per Lexicon._s:

Stem ends in Clitic Example
voiceless non-sibilant p t k f θ /s/ cat'skˈæts
sibilant s z ʃ ʒ ʧ ʤ /ᵻz/ boss'sbˈɑsᵻz
anything else /z/ today'stədˈAz

is Misaki's US form of the epenthetic vowel (ɪ is the british=True
form). This frontend loads the US lexicon and is present in the chain's
vocab.json, so the US form is the correct one here.

Scope, deliberately narrow

Faithful to stem_s, the rule fires only when the stem is a known word, so
an OOV stem leaves the token on the existing whole-word G2P path rather than
being re-shaped from a guess. Because the stem goes through the normal chain,
#775's hyphen split still applies underneath the clitic
(mother-in-law'smˈʌðɜɹ ɪn lˈɔz), and a glued entry that does exist
(it's) still wins, since stemming runs only after the miss.

Contractions are unaffected in practice — it's, he's, that's are lexicon
entries — and where one is not, the clitic rule gives the phonologically
identical result, since the is/has clitic and the possessive share the
-s phonology.

resolveWord / resolveHyphenatedCompound gain an allowFallback flag
(defaulting to today's behavior) to express "resolve, but only if known".

Only stem_s's 's branch is ported. Misaki's word.endswith("s'") and
trailing-' branches are unreachable here: splitWords only keeps an
apostrophe word-internally, so a trailing apostrophe is already emitted as its
own punctuation token. The es/ies plural branches are a separate change.

Tests

Adds coverage for voiced / voiceless / sibilant / vowel-final stems, curly
apostrophe, uppercase 'S with a case-sensitive stem, a hyphenated-compound
stem, OOV-stem fallback, lexicon-entry precedence, and a direct table check of
the clitic rule against Lexicon._s.

swift build, swift test --filter TTS (57 tests, incl. 40 phonemizer) and
swift-format lint are clean.

The Misaki lexicon stores the `-s` clitic as its own `'s` entry and carries
no glued possessive keys, so `today's` / `someone's` / `the boss's` miss
every lexicon tier in `KokoroAneEnglishPhonemizer.resolveWord` and the whole
inflected token reaches the BART G2P fallback, which mangles it
(`someone's` was heard as "Samian's" in a whisper transcription of the
rendered audio).

Python Misaki does not have this problem because `Lexicon.__call__` falls
through to `stem_s`: the stem is looked up on its own and the clitic phoneme
is appended by rule. This ports that rule.

After a full lexicon miss (and after the FluidInference#710 initialism spell-out), a token
ending in `'s` — case-insensitively, and post the FluidInference#774 apostrophe folding, so
`Today's`, `TODAY'S` and `Today\u{2019}s` all qualify — resolves its stem
through the normal chain minus the G2P fallback, then appends the clitic per
`Lexicon._s`:

  * `/s/`   after a voiceless non-sibilant (`p t k f θ`) — `cat's` -> `kˈæts`
  * `/ᵻz/`  after a sibilant (`s z ʃ ʒ ʧ ʤ`)            — `boss's` -> `bˈɑsᵻz`
  * `/z/`   otherwise                                    — `today's` -> `tədˈAz`

`ᵻ` is Misaki's US form of the epenthetic vowel (`ɪ` is the `british=True`
form); this frontend loads the US lexicon and `ᵻ` is present in the chain's
`vocab.json`.

Faithful to `stem_s`, the rule only fires when the stem is a *known* word, so
an OOV stem leaves the token on the existing whole-word G2P path rather than
being re-shaped from a guess. The stem going through the normal chain means
FluidInference#775's hyphen split still applies underneath the clitic
(`mother-in-law's` -> `mˈʌðɜɹ ɪn lˈɔz`), and a glued lexicon entry that does
exist (`it's`) still wins, because stemming runs only after the miss.

`resolveWord` / `resolveHyphenatedCompound` gain an `allowFallback` flag
(defaulting to today's behavior) to express "resolve, but only if known".

Tests: voiced / voiceless / sibilant / vowel-final stems, curly apostrophe,
uppercase `'S` with a case-sensitive stem, hyphenated-compound stem, OOV-stem
fallback, lexicon-entry precedence, and a direct table check of the clitic
rule against `Lexicon._s`.

`swift build`, `swift test --filter TTS` (57 tests) and `swift-format lint`
are clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant