Skip to content

fix(computer): smooth native drag trajectory - #3098

Draft
quanru wants to merge 7 commits into
mainfrom
fix/computer-linux-drag-trajectory
Draft

fix(computer): smooth native drag trajectory#3098
quanru wants to merge 7 commits into
mainfrom
fix/computer-linux-drag-trajectory

Conversation

@quanru

@quanru quanru commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • reproduce the Linux/VNC-style drag failure with a real Chromium window on Xvfb
  • move the held pointer through intermediate coordinates so native drag state machines can observe the active drag before release
  • add unit coverage for input ordering and a focused hosted-Linux regression job that needs no model credentials

Root cause

The computer driver moved from source to target in one native pointer jump. Desktop drag-and-drop state machines use the first movement beyond their threshold to activate dragging, so the target needs a later held-button movement before mouseup. With only one jump, the source entered drag mode but the target never observed it.

GitHub Actions reproduction

  • Before (same stabilized fixture, legacy one-jump implementation): failed
  • After (20-step held-pointer trajectory): passed

Validation

  • pnpm run lint
  • npx nx test @midscene/computer
  • npx nx build @midscene/computer
  • AI_TEST_TYPE=computer npx nx test @midscene/computer -- tests/ai/linux-drag-and-drop.test.ts --retry 0 (skipped locally on macOS; passed in the hosted Linux run above)

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-04T04:03:25.115963Z c164258 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@quanru
quanru marked this pull request as draft September 4, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant