Skip to content

Share exact UI element callouts with agents - #198

Merged
lastobelus merged 10 commits into
mainfrom
t3code/ui-element-callouts
Aug 21, 2026
Merged

Share exact UI element callouts with agents#198
lastobelus merged 10 commits into
mainfrom
t3code/ui-element-callouts

Conversation

@lastobelus

@lastobelus lastobelus commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Part of #196. Restacked onto main after #197 merged.

What changes

  • Option-clicking any element in a live development watcher pins a visible bounding box and copies one opaque mko-ui-v1: reference.
  • References contain a validated unique-ID anchor plus deterministic same-tag child indices, sibling counts, and structural fingerprints; they do not contain executable code or accept arbitrary JavaScript.
  • markover --instance dev element highlight <reference> resolves and highlights exactly that element in the addressed running instance.
  • markover --instance dev element clear removes the pinned callout.
  • Stale references fail with 404 and ambiguous anchors fail with 409, clearing the old overlay rather than highlighting another element.
  • The authenticated service route and picker are enabled only by the live development watcher. Release and non-watch instances expose neither workflow.

Done when

  • One Option-click pins an element and copies its stable reference.
  • The development CLI highlights and clears that reference in the same live window.
  • Stale and ambiguous references fail without selecting the wrong element.
  • Production and non-watch runtimes have no picker or local highlight route.
  • Automated and native two-way flow checks pass.

Excludes

  • General browser automation, remote debugging, or arbitrary renderer script execution.
  • Production inspection surfaces.
  • Visual changes owned by Improve Markover review UI #195.
  • Compatibility behavior for unreleased prototype references.

Validation

  • npm run ci:local: pass, 842/842 tests plus smoke.
  • Focused element/service/IPC/CLI/layout/watcher tests: 116/116 pass.
  • Native QA on the PR 198 watcher:
    • one Option-click on Open Markdown copied the opaque reference anchored to empty-open-button;
    • the CLI resolved it to {x:527, y:475, width:126, height:36} and the visible pink box labeled button#empty-open-button enclosed that exact button;
    • element clear removed the overlay;
    • a valid reference for a missing anchor failed as stale with exit status 1 and left no overlay.

@lastobelus
lastobelus marked this pull request as ready for review August 21, 2026 18:29

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 36e30940fc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/development-element.ts Outdated
Comment thread src/development-element.ts
@lastobelus
lastobelus force-pushed the t3code/stable-ui-dev-workflow branch from 82b141d to a124443 Compare August 21, 2026 18:49
@lastobelus
lastobelus force-pushed the t3code/ui-element-callouts branch from 36e3094 to 1403f91 Compare August 21, 2026 19:10
@lastobelus
lastobelus changed the base branch from t3code/stable-ui-dev-workflow to main August 21, 2026 19:10
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1403f91446

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/markover.ts Outdated
Comment thread src/development-element.ts
Comment thread src/development-element.ts
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2e56fc0702

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/renderer.ts Outdated
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f37dbdab53

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/development-element.ts
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7850d4f1e1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/development-element.ts
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e2d11cc676

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/development-element.ts Outdated
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4a4400e5ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/development-element.ts Outdated
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1868ea5c73

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/development-element.ts Outdated
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b08d39d8d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/development-element.ts
@lastobelus

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 580d8e1ee8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@lastobelus
lastobelus merged commit 2d26164 into main Aug 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant