Skip to content

Cleanup guard: no flipped negations, "yes" pastes, numbers and vocabulary fixes accepted - #16

Merged
radiumcoders merged 18 commits into
masterfrom
guard-fixes
Nov 11, 2026
Merged

radiumcoders merged 18 commits into
masterfrom
guard-fixes

Conversation

@radiumcoders

Copy link
Copy Markdown
Owner

Fixes to the local cleanup's faithfulness guard (crates/core/src/cleanup.rs), from testing master after #11. Built on #12; its two xtask commits drop out of this diff once #12 merges. The Claude Haiku path is untouched.

  • A dropped (or added) negation is unfaithful however little else changed: "do not merge this" → "Merge this." and "the test is not passing" → "The test is passing." used to pass. Self-corrections ("at three, no, four" → "at four") still do.
  • "yes", "ok", "okay", "yeah"… on their own paste ("Yes.") instead of being dropped as an aside.
  • Numbers written as digits are faithful: 120 (with or without "and"), 2.5, 2026, 31st, 404, phone digits, 9:30. Before, these pasted the raw transcript.
  • Your words.txt explains the fixes it asks for: "cube and eighties" → "Kubernetes" passes where the hint says it's heard that way.
  • A new word must match what was said at that position, not merely resemble any word: "Dictation" → "Diction" and "Py script" → "Python script" are now caught.
  • Spoken emails/URLs ("jay at example dot com") equal the written ones.

localflow --eval: 32/41 before and after, identical per-case verdicts. cargo test --workspace: 106 passed. Clippy clean.

…as asides

The cleanup model answers <aside> for a one-word reply, and a short aside
was always believed, so nothing was pasted. A dictation of one or two
reply words (yes, ok, okay, yeah, thank you, ...) is never an aside now:
it goes in as a sentence ("Yes."). "hmm let me think" is still left out.
…else changed

"the test is not passing" -> "The test is passing." and "do not merge
this" -> "Merge this." passed the guard, because fewer than three words
were dropped. A dropped not/no/never/don't/can't/without/... now counts on
its own, unless what it negated went with it ("it's not good, it's
great" -> "It's great."), it was said twice, or "no" takes back what
came before it ("at three, no, four" -> "at four"). A negation the
speaker never said is caught the same way.
…, 404, 9:30)

The guard only knew simple number words, so correct formatting was
thrown away for the raw transcript: "one hundred twenty" -> 120 counted
as three dropped words, and 2.5, 2026, 31st and 404 as made up. The
spoken-number check now also reads "and" after a scale, decimals ("two
point five"), words read out in a row ("twenty twenty six", "four oh
four", phone numbers), ordinals, "one and a half" and clock times
("half past nine", "quarter to nine", "nine o'clock"), and remembers
which words each reading stands for, so number words the cleanup wrote
as digits aren't counted as dropped (and the ones it didn't still are:
"five five five one two three four" -> 5:55 is caught). Thousands
separators and "st/nd/rd/th" are ignored. Runs are capped at 20 words
and the sums saturate.
"send it to jay at example dot com" -> "Send it to jay@example.com."
was flagged as dropping "example dot com": everything after an @ was
taken for a file reference and left out of the comparison. The @ of an
email address now counts as the "at" that was said, and symbols said as
words (dot, slash, colon, dash, ...) aren't counted as dropped.
The faithfulness guard never looked at words.txt, so with 'Kubernetes:
often heard as "cube and eighties"' in it, "deploy it to cube and
eighties tonight" -> "Deploy it to Kubernetes tonight." was flagged as
adding "kubernetes" and the transcript was pasted as heard. The guard
now gets the vocabulary: a term's words are explained where one of the
ways it's misheard was said, or the term itself spelled or spaced
another way ("next JS" -> Next.js), and the misheard words aren't
counted as dropped. Being in the vocabulary isn't enough on its own:
the model writes its terms where nothing like them was said ("slash got
shaders skill" -> "Claude Code"), and that is still caught.
…some word said

explained() took any new word 60% like any word of the transcript, or
sharing its first four letters, so "Dictation" -> "Diction" passed, and
"Py script" -> "Python script" because of a "then" elsewhere in the
dictation (and "computer" -> "company"). A new word is now explained by:
the word itself or another form of it (ship / shipping / shipped, by its
ending) anywhere in what was said; and only next to where it was
written, the same word a letter apart (colour / color, there / their; a
short word may only gain or lose one, so "test" isn't "text"), words
said apart or run together ("post gress" -> Postgres, "nextjs" ->
Next.js), casual speech written out ("gonna" -> going to), a word that
sounds the same (their / they're), or a vocabulary term. "Claw" ->
Claude now rests on words.txt saying so, which it does.
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 29 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 70f43714-a782-4743-b909-a5be58b0f908
Commits

Reviewing files that changed from the base of the PR and between 2a9a171 and 9fd03e0.

Files selected for processing: 3

  • crates/core/src/cleanup.rs
  • crates/core/src/vocab.rs
  • words.txt
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@radiumcoders

Copy link
Copy Markdown
Owner Author

Review fixes (independent review, since CodeRabbit is at its limit), 11 commits:

  • An invented negation is caught even next to a said one ("I can't believe it works" → "…it doesn't work.").
  • "dont"/"cant"/"isnt" and "do not"/"cannot" are the contractions they stand for.
  • A leading filler "no" ("yeah no I think it's fine") isn't a dropped negation; "no errors so far" → "Errors so far." still is.
  • A dropped negation is unfaithful whatever went with it: "it does not work" → "It works.", "I did not merge it" → "I merged it." are now caught. Excused only when said twice, followed by a dropped correction word, or the sentence starts over.
  • "cloud" is a way Claude is heard (words.txt), so "ask cloud to fix the bug" → "Ask Claude…" passes again; "deploy it to the cloud" keeps "cloud".
  • "one point five million" → "1.5 million" passes; "count one two three" → "Count 6.", "give me one second" → "Give me 3.", "ninety nine point nine" → "9.9" are caught.
  • A reply over muttering is just the reply; "OK," → "OK."

localflow --eval: 32/41, identical per-case verdicts and outputs. Tests and clippy pass.

@radiumcoders
radiumcoders merged commit 27c964e into master Nov 11, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant