Support for non alphabetical unicode characters

[nisanthchunduru]

Hi,

I've added support for some non alphabetical unicode characters like currency characters, math characters, symbols etc.

Previously, non alphabetical unicode characters were gobbled up

[2] pry(main)> utf8_html
=> "＋<br />ー<br />〜<br />＝<br />─<br />a　(with double-byte space)<br />￥<br />＆<br />％<br />＃<br />＄<br />！<br />？<br />＞＜<br />・<br />／<br />「」<br />＠<br />、。"
[3] pry(main)> TruncateHtml::HtmlTruncator.new(TruncateHtml::HtmlString.new(utf8_html), length: 1000).truncate
=> "<br />ー<br />〜<br /><br /><br />a(with double-byte space)<br /><br />＆<br />％<br />＃<br /><br />！<br />？<br /><br />・<br />／<br />「」<br />＠<br />、。"

Now

[2] pry(main)> utf8_html
=> "＋<br />ー<br />〜<br />＝<br />─<br />a　(with double-byte space)<br />￥<br />＆<br />％<br />＃<br />＄<br />！<br />？<br />＞＜<br />・<br />／<br />「」<br />＠<br />、。"
[3] pry(main)> TruncateHtml::HtmlTruncator.new(TruncateHtml::HtmlString.new(utf8_html), length: 1000).truncate
=> "＋<br />ー<br />〜<br />＝<br />─<br />a　(with double-byte space)<br />￥<br />＆<br />％<br />＃<br />＄<br />！<br />？<br />＞＜<br />・<br />／<br />「」<br />＠<br />、。"

[crv]

This is useful. @nisanth074 Can you also add temperature degree symbol (e.g. °C) in the list of your special characters ?

[wwcline]

This might (at least partly) resolve #35.

[rob-mcgrail]

Is this going to be merged?