Skip to content

web: Map pointer coords through getBoundingClientRect#23659

Open
evil1morty wants to merge 2 commits intoruffle-rs:masterfrom
evil1morty:web-pointer-coords-css-transform
Open

web: Map pointer coords through getBoundingClientRect#23659
evil1morty wants to merge 2 commits intoruffle-rs:masterfrom
evil1morty:web-pointer-coords-css-transform

Conversation

@evil1morty
Copy link
Copy Markdown
Contributor

@evil1morty evil1morty commented May 6, 2026

Description

Pointer events arriving at the SWF stage are mis-mapped whenever the canvas (or any ancestor) is rendered at a different visual size than its layout box — most commonly under the CSS zoom property, but also under CSS transforms. Concretely, with zoom: 0.5 on a parent of the player, clicks register at the wrong stage location.

The current code in web/src/lib.rs is:

x: js_event.offset_x() * instance.device_pixel_ratio,
y: js_event.offset_y() * instance.device_pixel_ratio,

Two things break:

  1. The canvas backing buffer is sized from canvas.client_width() times device_pixel_ratio. Under zoom: 0.5 the layout box is unchanged at e.g. 1000 CSS px and the backing buffer is 1000 px — but the canvas is visually 500 px on screen.
  2. MouseEvent.offsetX on Chromium reports the position relative to the visually rendered rect, not the untransformed padding edge. Multiplying that by device_pixel_ratio gives a value scaled to 0..(visual width × DPR), which doesn't match the 0..(layout width × DPR) the backing buffer expects.

This PR replaces offsetX/Y * device_pixel_ratio with a small helper that computes coordinates from clientX/Y and getBoundingClientRect(). The rect reflects whatever CSS zoom / transforms / positioning the canvas ends up with, and the backing buffer is canvas.width × canvas.height. The ratio (client - rect.origin) * canvas_size / rect_size is therefore correct under any combination of:

  • CSS zoom on the canvas or any ancestor
  • CSS transforms (transform: scale, etc.)
  • Browser zoom
  • Device pixel ratio
  • CSS positioning / margins / padding

Three call sites updated (pointermove, pointerdown, pointerup); the wheel event doesn't pass a position so it's untouched.

Element::get_bounding_client_rect returns a DomRect, so DomRect is added to the web-sys feature list.

Testing

Reproducing the original bug:

  1. Visit https://safariislandsgame.com/play/?lang=en in Chromium and use the in-page - / + zoom buttons — these set CSS zoom on a wrapper. Clicks register offset from where the user clicks.
  2. Or, in DevTools on any page, set zoom: 0.7 on a parent of <ruffle-player>. Same result.

After this PR, clicks land at the cursor in both cases. Also verified with browser zoom at 75% / 100% / 125% / 150%, and transform: scale (which already happened to map correctly via offsetX, but is now consistent with the zoom path through the same helper).

cargo fmt --all clean. cargo clippy -p ruffle_web --target wasm32-unknown-unknown zero warnings.

No new automated test added — couldn't find an existing harness for pointer-event mapping under DOM transforms; happy to add one if reviewers point me at a fixture pattern.

Checklist

  • I, a human, have self-reviewed this PR and fully understand the changes within.
  • I have made or updated tests where possible.
  • All of my commits are properly scoped, compile successfully, and pass all tests.
  • This PR does not make sense to split up into smaller PRs.
  • An LLM was involved in the authoring of this code.

`MouseEvent.offsetX/Y * devicePixelRatio` produces wrong stage
coordinates whenever the canvas (or any ancestor) has a CSS transform
applied — `transform: scale(0.5)` on a parent shrinks the visual rect
but leaves `client_width` (the layout box used to size the canvas
backing buffer) untouched, while `offsetX` reflects the visual position
on Chromium. Click positions register at the wrong stage location.
Browser zoom worked because zoom changes layout uniformly.

Compute `(client_x - rect.left) * canvas.width / rect.width` (and the
y equivalent) so the stage coord is always derived from the visual rect
and the backing-buffer size, which together describe the actual pixel
mapping regardless of CSS transforms, browser zoom, or DPR.

Adds the `DomRect` web-sys feature for `Element::get_bounding_client_rect`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@danielhjacobs danielhjacobs added A-web Area: Web & Extensions T-fix Type: Bug fix (in something that's supposed to work already) llm The PR contains mostly LLM-generated code labels May 6, 2026
@kjarosh
Copy link
Copy Markdown
Member

kjarosh commented May 6, 2026

Can you add an integration test for it under web/packages/selfhosted/test/integration_tests so that each callback is tested?

@evil1morty
Copy link
Copy Markdown
Contributor Author

Added an integration test under web/packages/selfhosted/test/integration_tests/pointer_coords:

  • 200x200 SWF stage wrapped in a transform: scale(0.5) parent, so the canvas's visual rect is 100x100 while its backing buffer stays 200x200.
  • AS3 Test.as listens for MOUSE_DOWN, MOUSE_UP and MOUSE_MOVE on stage and traces the rounded (stageX, stageY).
  • test.ts dispatches synthetic PointerEvents on the canvas at known clientX/Y inside the visual rect; the trace asserts the resulting stage coords. Under the previous offset_x * dpr mapping each case would land off by a factor of 2; with the new ratio mapping they land on the expected coordinates.

Three cases — one per callback (pointerdown/up/move) — to satisfy the request that each one be exercised.

@evil1morty evil1morty force-pushed the web-pointer-coords-css-transform branch from fc986fd to c3f08b8 Compare May 6, 2026 20:50
@evil1morty
Copy link
Copy Markdown
Contributor Author

The CI logs show the pointer-coord values were exactly right — onMouseDown(100,100), onMouseUp(0,0), onMouseMove(50,150) all traced as expected. The failure was that I missed the synthetic MOUSE_MOVE Ruffle dispatches before each MOUSE_DOWN / MOUSE_UP to settle the cursor at the new position (Flash semantics). My single-element expected arrays consumed the move, then the next assertion saw the down/up still queued.

Force-pushed an amend that updates each expected list to include the leading move:

  • pointerdown: ["onMouseMove(100,100)", "onMouseDown(100,100)"]
  • pointerup: ["onMouseMove(0,0)", "onMouseUp(0,0)"]
  • pointermove: unchanged

That matches the trace order the test framework already saw in the failed run.

The host page wraps a 200x200 SWF in a `transform: scale(0.5)` parent,
so the canvas's visual rect is 100x100 while the backing buffer stays
200x200. Each pointer callback is exercised with a synthetic
PointerEvent dispatched at known clientX/Y inside that visual rect, and
the AS3 stage handlers trace the resulting (stageX, stageY). The
expected stage coordinates assume the new ratio-based mapping in
`pointer_to_stage_coords` — under the previous `offset_x * dpr`
implementation each test would land off by a factor of 2.

Covers pointermove, pointerdown, and pointerup — the three callbacks
the fix touches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@evil1morty evil1morty force-pushed the web-pointer-coords-css-transform branch from c3f08b8 to 00b9f29 Compare May 6, 2026 21:00
@n0samu
Copy link
Copy Markdown
Member

n0samu commented May 7, 2026

How did you notice this issue? Are you scaling the Ruffle canvas on your website?
(this is a question for the human behind this PR, not Claude)

@evil1morty
Copy link
Copy Markdown
Contributor Author

I noticed it while testing on https://safariislandsgame.com/play/?lang=en
I used the site's built-in zoom buttons (- and + in the menu) and saw the misalignment
I've now deobfuscated their zoom logic (https://safariislandsgame.com/play/test.js) here's what they actually do:

$("#all").css("zoom", scale);
$("#all").css("-moz-transform", "scale(" + scale + "," + scale + ")");
$("#all").css("MozTransform", "scale(1)");
$("#all").css("MozTransformOrigin", "center");

So on Firefox they use transform: scale(...), and on Chromium they use the zoom property
The transform: scale path works fine, the issue only reproduces with the zoom rule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-web Area: Web & Extensions llm The PR contains mostly LLM-generated code T-fix Type: Bug fix (in something that's supposed to work already)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants