Support for non alphabetical unicode characters - #58

Open

nisanthchunduru wants to merge 10 commits into

hgmnz:masterfrom

SupportBee:better_unicode_support

Commits on Dec 26, 2014

Split token regex into understandable chunks
nisanthchunduru
committed
2037836
Add test to ensure non alphabetic unicode characters are not gobbled up
nisanthchunduru
committed
954a66a
Add support for unicode currency characters
nisanthchunduru
committed
bd7ddbf
Add support for unicode math characters
nisanthchunduru
committed
8cbfff1
Add support for unicode space characters
nisanthchunduru
committed
350712a
Add support for other unicode symbols
nisanthchunduru
committed
3d2591a
Refactor token regex
nisanthchunduru
committed
e3b3efa
Add comments explaining the token regex
nisanthchunduru
committed
c950cf7

Commits on Dec 28, 2014

Extract valid tag body characters into a separate variable
nisanthchunduru
committed
9000c42

Commits on May 18, 2015

Adds support for HalfWidth and FullWidth forms of unicode charecters
avinasha
committed
bf787cc