Fix Unicode regex patterns to support international characters

[gardner]

Fix Unicode Regex Patterns to Support International Characters

Fixes #1420

Summary

• Updated MastodonRegex.swift to replace ASCII-only character classes [a-zA-Z0-9_] with Unicode character classes [\\p{L}\\p{M}\\p{N}_] in both highlightPattern and autoCompletePattern regex patterns
• This aligns with the pattern used for hashtags in

  mastodon-ios/MastodonSDK/Sources/MastodonUI/Scene/ComposeContent/Helper/MastodonRegex.swift

  Line 31 in 19b6947

  let word = "\\p{L}\\p{M}\\p{N}\\p{Pc}"

• Added comprehensive test cases in MastodonRegexTests.swift to verify the regex patterns correctly match Unicode characters with combining marks and other international characters
• This ensures proper matching of usernames and domains with Unicode characters, such as café@Examplë.com, German umlauts, Chinese characters, and Japanese characters

Problem

The current regex patterns in MastodonRegex.swift were using ASCII-only character classes [a-zA-Z0-9_] which don't properly match international Unicode characters. This meant that usernames and domains containing characters with combining marks (like café or examplë.com) or other Unicode characters (like Chinese or Japanese characters) would be cut off prematurely, as these characters were not being recognized as valid parts of the username or domain.

Solution

Updated both regex patterns to use comprehensive Unicode character classes:

• highlightPattern: Changed from @([a-zA-Z0-9_]+) to @([\\p{L}\\p{M}\\p{N}_]+)
• autoCompletePattern: Changed from @([a-zA-Z0-9_]+) to @([\\p{L}\\p{M}\\p{N}_]+)

The new character classes include:

• \\p{L} - Unicode letters from any language
• \\p{M} - Combining marks (accents, umlauts, etc.)
• \\p{N} - Unicode numbers
• _ - Underscore (preserved from original pattern)

This change allows the regex to properly recognize a much broader range of characters as valid parts of usernames and domains, including but not limited to:

• Characters with combining marks: é, ë, ü, ñ, etc.
• Characters from other languages: Chinese, Japanese, Arabic, etc.
• All previously supported ASCII characters

Tests

Added comprehensive test suite in MastodonRegexTests.swift with test cases including:

• @café@examplë.com (characters with combining marks)
• @unikonstanz@bawü.social (German umlauts)
• @user@täst.de (another example with umlauts)
• @björn@exämple.com (multiple combining marks)
• @用户@example.com (Chinese characters)
• @example@例え.テスト (Japanese characters)
• Various ASCII test cases to ensure existing functionality still works

All tests verify that the regex patterns correctly match the full username and domain, including all Unicode characters.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}