Skip to content

feat: add macOS support#551

Open
Zerlight wants to merge 15 commits into
shinyflvre:mainfrom
Zerlight:main
Open

feat: add macOS support#551
Zerlight wants to merge 15 commits into
shinyflvre:mainfrom
Zerlight:main

Conversation

@Zerlight

@Zerlight Zerlight commented May 26, 2026

Copy link
Copy Markdown

Summary

This PR adds baseline macOS compatibility while preserving the existing Windows behavior as much as possible. The focus is compile/runtime safety, platform-correct storage, model import support, macOS startup support, app icon handling, HiDPI-friendly window sizing, and initial macOS sit-on-window support.

What Changed

  • Isolated Windows-only runtime integrations behind platform checks.
  • Added platform-aware app storage through MateEnginePaths.
    • Uses Unity persistent/cache paths for internal runtime files.
    • Copies imported .vrm / .me files into managed app storage on macOS.
    • Preserves the existing Windows custom data directory behavior.
  • Restored macOS model import.
    • Replaces the broken macOS standalone picker path.
    • Allows .vrm / .me selection through code-side validation.
  • Added a cross-platform desktop window API foundation.
    • Adds a macOS CoreGraphics/AppKit native plugin.
    • Moves shared scripts away from direct Win32-only assumptions where needed for compile safety.
  • Added macOS sit-on-window support.
    • Reuses the existing Windows sitting algorithm where possible.
    • Adds platform wrappers for cursor position, own window rect/move, target window rects, z-order checks, topmost state, and occlusion handling.
    • Uses CoreGraphics/AppKit window metadata on macOS.
    • Does not request Screen Recording, Accessibility, or Input Monitoring permissions.
    • Does not use ScreenCaptureKit, window image capture, or AX APIs.
    • Adds native-side caching and lightweight follow smoothing to reduce macOS window metadata polling overhead.
    • Keeps existing Windows sitting/taskbar behavior on the Win32 path.
  • Added macOS LaunchAgent autostart support.
    • Keeps Windows registry autostart unchanged.
  • Added macOS app icon support.
    • Includes Icon Composer assets and pre-Tahoe PNG fallback assets.
    • Adds a macOS-only editor postprocessor for Xcode / .app icon setup.
    • Note that this icon is a homebrew one since I cannot find the vector icon for Mate-Engine
  • Adjusted macOS HiDPI window sizes.
    • Keeps Windows size presets unchanged.
    • Uses smaller logical sizes on macOS so Retina displays do not start oversized.
  • Added macOS menu bar item support aligned to Windows SystemTray menu

What Is Not Supported Yet

  • Dock sitting is not implemented.
  • Launchpad-specific hiding behavior is not included.
  • ScreenCaptureKit is not included.
  • This does not provide full macOS window management across Spaces, fullscreen apps, minimized windows, or hidden windows.
  • This does not add window content capture or screenshot/streaming support.
  • This does not change Windows taskbar sitting behavior.
  • Window position persistence is not included.
  • macOS sit-on-window is best-effort and metadata-based; behavior may vary with special system windows, fullscreen Spaces, or hidden/minimized apps.

Issues about Shaders

For some reasons, certain shaders will not be included, or properly processed/built on macOS. Taking example of my own model, macOS build will not display outline shaders. I am currently unable to handle with this issue since I am not a Unity expert.

Validation

  • Changes are split by feature area and scoped to macOS compatibility.
  • Windows-specific behavior is guarded to preserve existing Windows functionality.
  • macOS basic functionality has been verified.
  • macOS sit-on-window has been manually tested against normal desktop app windows.
  • macOS sitting works without Screen Recording, Accessibility, or Input Monitoring permission prompts.
  • Windows has not been fully regression-tested yet.

Preview

image image

@Zerlight

Copy link
Copy Markdown
Author

macOS Sit-on-Window Performance Notes

Summary

The macOS sit-on-window implementation is expected to be more expensive than the Windows path. The bottleneck is not the sitting animation itself, but how macOS exposes desktop window metadata when we avoid Screen Recording, Accessibility, Input Monitoring, and window-content APIs.

Windows can query and move specific HWNDs with cheap Win32 calls such as GetWindowRect, SetWindowPos, and z-order APIs. On macOS, the no-permission path mainly relies on CoreGraphics window metadata snapshots from CGWindowListCopyWindowInfo, plus AppKit for the MateEngine window itself. That means target rect, alive checks, z-order, and occlusion data often come from a full on-screen window snapshot rather than a direct per-window handle query.

Main Bottlenecks

  • CoreGraphics window snapshot cost

    • CGWindowListCopyWindowInfo returns metadata for many on-screen windows at once.
    • There is no equally cheap no-permission equivalent of Win32 GetWindowRect(hwnd) for arbitrary app windows.
    • Rebuilding this snapshot too often can quickly dominate CPU and hit WindowServer/Unity main-thread scheduling limits.
  • Polling instead of event-driven updates

    • Without Accessibility observers or private APIs, MateEngine cannot receive reliable move/resize events for other apps' windows.
    • The sitting code must poll target window rects while snapped.
    • Higher polling rates improve visual follow smoothness but scale CPU cost sharply.
  • Occlusion and z-order

    • To make the pet appear behind the target window and higher windows, the implementation needs target rects, occluder rects, and z-order checks.
    • On macOS these are snapshot-backed operations, so the cost grows with the number of visible windows and displays.
  • Coordinate conversion

    • CoreGraphics window metadata and AppKit NSWindow / NSScreen APIs use different coordinate conventions.
    • Multi-monitor and Retina displays add point/pixel and origin-offset handling.
    • This is mostly a correctness risk, but it also forces additional native-side normalization.
  • Unity window movement

    • The pet window is still moved every frame while sitting/following.
    • AppKit window movement is stable, but it is not as cheap as the direct Win32 path and must stay synchronized with Unity's update loop.

Current Mitigations

  • macOS window metadata is cached in the native bundle instead of repeatedly scanning from C#.
  • Snapshot refresh is throttled with a short TTL, currently around 1 / 30s.
  • The C# sitting algorithm keeps its existing windowEnumFPS cadence for full window enumeration.
  • Target-window rect prediction smooths motion between CoreGraphics snapshot refreshes.
  • MateEngine's own window rect/move path uses AppKit directly to avoid UniWindow coordinate mismatch.
  • Topmost calls are deduplicated natively.
  • The Windows implementation remains on the original Win32/PInvoke path.

Why Not Use Permissions Here

Accessibility and Screen Recording could unlock other approaches, but they are intentionally not part of this PR:

  • Screen Recording is not needed because we only need metadata, not pixels.
  • Accessibility introduces a system permission prompt and changes the product/security surface.
  • Earlier AX-based experiments were not clearly better in practice and could produce less stable target-window rect updates.
  • Keeping the feature no-permission makes it more suitable for upstreaming into a previously Windows-only project.

Practical Expectations

The macOS implementation should be usable, but it may not match Windows CPU usage. Windows has direct, mature, per-window APIs for this workflow; macOS requires snapshot-based polling for no-permission window metadata.

The most important tuning tradeoff is:

  • Lower snapshot/enumeration frequency: lower CPU, less responsive follow.
  • Higher snapshot/enumeration frequency: smoother follow, much higher CPU / WindowServer pressure.

The current implementation chooses a conservative middle ground: throttled snapshots, prediction between updates, and no additional privacy prompts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant