Skip to content

Use UTF-16 positions in the markup language server#159

Open
narmi924 wants to merge 1 commit into
vercel-labs:mainfrom
narmi924:fix/lsp-utf16-positions
Open

Use UTF-16 positions in the markup language server#159
narmi924 wants to merge 1 commit into
vercel-labs:mainfrom
narmi924:fix/lsp-utf16-positions

Conversation

@narmi924

@narmi924 narmi924 commented Jul 19, 2026

Copy link
Copy Markdown

Summary

  • map incoming LSP UTF-16 positions to UTF-8 byte offsets for completion and hover
  • map parser byte columns back to UTF-16 diagnostic ranges
  • clamp CRLF line ends and malformed or out-of-range positions safely
  • advertise UTF-16 position encoding and cover CJK, emoji, and protocol round trips

Why

The markup parser tracks byte columns, while LSP clients use UTF-16 code units by default. The server previously treated protocol character values as byte offsets and emitted parser byte columns directly, so CJK text or emoji earlier on a line shifted completion, hover, and diagnostic locations.

Validation

  • direct markup LSP suite: all 10 tests passed
  • zig build test-markup-cli
  • zig build validate
  • scripts/gate.sh fast origin/main: validation and all frontend, native, and mobile example groups passed on this branch; root tests reached only the existing Windows CRLF source-pattern failures fixed by Make build validation portable on Windows #157
  • detached combined validation with Make build validation portable on Windows #157: root tests, validation, and all frontend/native/mobile example groups passed

@vercel

vercel Bot commented Jul 19, 2026

Copy link
Copy Markdown

@narmi924 is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

@narmi924
narmi924 marked this pull request as ready for review July 19, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant