Support /\p{General_Category=Letter}/gu?

[crapthings]

will this support

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions/Unicode_Property_Escapes#general_categories

let regexp = /\p{sc=Han}/gu; // returns Chinese hieroglyphs

let str = `Hello Привет 你好 123_456`;

alert( str.match(regexp) ); // 你,好

[hildjj]

Maybe. See #15 for the precursor to this discussion.

[hildjj]

I was thinking about this some more, and I bet we could do something cool with unicode-trie. It wouldn't be quite as fast as something the JS engine provides, but it's the basic approach that ICU uses in most JS engines. The idea would be:

• Either
  • use a prepackaged trie (like unicode-properties but brought up to date)
  • package one with just the properties we need (at least Script, Script_Extensions, and Category). I've done something similar for basura.
• parse a codepoint at a time
• Create rules that have the form Nd = d:. &{ return properties.getCategory(d) === 'Number' }
  • With good benchmarking, you might add [0-9] / to the front of that, or even exclude the normal ascii range before doing the lookup, to avoid the trie dip if you don't need it
• Make it easy to reuse these rules by
  • adding the ability to import grammars
  • packaging these rules and the properties into a separate package that would have a runtime component that includes the trie
• Provide some syntactic sugar that is more like the JS you have above
• Update the peggy grammar to use this runtime, removing the blob of BMP-only regex from the middle.

[hildjj]

For reference, this works, but I don't know how fast it is:

const peggy = require('peggy')

const {parse} = peggy.generate(`
{{
const props = require('unicode-properties')
function toInt(str) {
  return [...str].reduce((t, v) => (t * 10) + props.getNumericValue(v.codePointAt(0)), 0)
}
}}
Numbers = head:Number tail:(" " @Number)* { return [head].concat(tail) }
Number = num:$Nd+ { return toInt(num) }
Nd
  = [0-9]
  / d:[^\u0000-\u065f] &{ return props.isDigit(d.codePointAt(0)) }
`)

console.log(parse('123١6 ੨੫੯')) // [ 12316, 259 ]

[hildjj]

More discussion in #375