test(browser): guard archive expression locale labels against silent drift - #416
Open
OfficialAbhinavSingh wants to merge 2 commits into
Open
Conversation
…drift The conversation-menu, archive-item, unarchive, and confirmation-toast matchers in buildArchiveConversationExpression each carry their own per-locale label list (steipete#407 added Japanese, earlier work added Polish). Nothing asserted the full set together, so a future edit could drop or mistype an existing locale's label without any test noticing — it would only surface as a live-account regression report, the exact gap named in docs/browser-mode.md's "Model picker drift" note. Add one inventory test per matcher enumerating every label it currently relies on. Verified these actually catch drift: temporarily removed one Polish literal from the source and reran — the new test failed with the literal it expected, confirmed, then restored the source unchanged. No production code touched.
Contributor
|
🦞👀 Pull request received. I will update this pull request when review starts. |
clawsweeper
Bot
added
merge-risk: 🚨 other
Contributor
|
Codex review: needs real behavior proof before merge. Reviewed August 25, 2026, 6:19 AM ET / 10:19 UTC. ClawSweeper reviewWhat this changesThe PR adds test inventories for each localized-label matcher in Oracle’s ChatGPT archive-conversation browser flow. Merge readiness⛔ Blocked until real behavior proof from a real setup is added - 3 items remain Keep open: the follow-up fixes the prior matcher-scope false negative and no remaining code defect was found, but the external-PR real-behavior proof gate is still unmet. Priority: P3 Review scores
Verification
Live VerificationCommand: Result: PASS (completed) Assertions:
How this fits togetherOracle serializes archive-control matching logic into a ChatGPT browser page to locate menus, archive actions, confirmation controls, and success messages. These tests inspect that generated matcher expression so removal of known locale labels fails during validation. flowchart LR
A[ChatGPT localized controls] --> B[Archive browser action]
B --> C[Generated page matcher]
C --> D[Localized label lists]
D --> E[Archive menu and confirmation]
F[Inventory tests] --> C
E --> G[Archived conversation]
Before merge
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Keep the focused test and add a redacted browser trace showing a localized archive control reaches a successful archive result, then re-review this head. Do we have a high-confidence way to reproduce the issue? Not applicable: this PR adds preventive regression coverage rather than repairing a presently reported runtime failure. Is this the best way to solve the issue? Yes: scoped matcher slices and quoted-literal assertions directly address the earlier duplicate-label and substring false negatives. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 79e483bd9dc8. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (1 earlier review cycle)
|
clawsweeper caught a real gap: the inventory tests added in this PR
checked each label against the whole serialized expression, not the
specific matcher function that consumes it. Several labels ("archive",
"アーカイブ") are legitimately repeated across findArchiveMenuItem and
findArchiveConfirmationButton, so a global toContain(label) stayed
green even when a label was deleted from just one of them.
Slice the expression per matcher function before asserting, and match
each label as a quoted literal ('label') rather than a bare substring
- "アーカイブ" is itself a substring of "アーカイブする", so a bare
check would still pass after the standalone literal is removed as long
as the longer one survives.
Verified against exactly clawsweeper's scenario: removed the 'アーカイブ'
exact-match line from findArchiveConfirmationButton only (left it in
findArchiveMenuItem). First attempt (unquoted slice check) stayed
green - false negative, confirming the deeper substring issue above.
Fixed to quoted matching, reran: failed on exactly that label. Restored
the source, confirmed green again (14/14).
clawsweeper
Bot
added
rating: 🦪 silver shellfish
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
buildArchiveConversationExpression).Problem
docs/browser-mode.mdnames this gap directly:The archive matcher is the clearest recent example. It carries four separate per-locale label lists (menu "more options", archive item/confirm, unarchive/restore exclusion, confirmation toast), each independently extended locale-by-locale (#407 added Japanese; Polish predates that). Nothing asserted the full set together — only a handful of Japanese literals were spot-checked. A future PR touching any one of these
.includes(...)chains could silently drop or mistype an existing locale's label, and it would only surface later as a live-account regression report.Change
tests/browser/archiveConversation.test.ts: one inventory test per matcher (menu, archive action, unarchive exclusion, confirmation toast), asserting every locale label currently inbuildArchiveConversationExpressionis present. A drop or typo in any of them now fails CI at review time instead of live.Update: scoped assertions per matcher (addresses review feedback)
The first version checked each label against the whole serialized expression. Several labels ("archive", "アーカイブ") are legitimately repeated across
findArchiveMenuItemandfindArchiveConfirmationButton, so a globaltoContain(label)stayed green even when a label was deleted from just one of them — reviewed and flagged correctly.Fixed: the expression is now sliced per matcher function before asserting, so a check can only be satisfied by that matcher's own labels. Labels are also matched as quoted literals (
'label') rather than bare substrings — "アーカイブ" is itself a substring of "アーカイブする", so a bare check would still pass after the standalone literal is removed as long as the longer one survives.Verified against exactly the flagged scenario: removed the
'アーカイブ'exact-match line fromfindArchiveConfirmationButtononly (left it infindArchiveMenuItem). First attempt (unquoted slice check) stayed green — a false negative, confirming the substring issue above. Fixed to quoted matching, reran: failed on exactly that label. Restored the source, confirmed green again (14/14).Original drift-catch proof (still holds against the current version): temporarily removed one Polish literal (
opcje), reran, restored:git diff --stat src/browser/actions/archiveConversation.ts→ no diff after both restores; production code was never touched.Checks
pnpm vitest run tests/browser/archiveConversation.test.ts— 14 passedpnpm test— 152 files / 1795 passed / 44 skipped (unchanged skip count)pnpm run typecheck— cleannpx oxlint tests/browser/archiveConversation.test.ts— cleannpx oxfmt --check— cleanCHANGELOG.mdleft as-is (mis-formatted on main; release owner per #349).Scope note
This is a pilot on one file to establish the pattern before extending it —
thinkingTime.ts(behind #405's effort-label locale fixes) and the other action files with per-locale matchers are natural next candidates, as follow-up PRs.