In the process of upgrading from 1.8 to 1.9 we are getting a lot of warnings about "Ruby 1.9 doesn't support Unicode normalization yet". However the commit that added those lines is from 2008 and just mentions "Ruby 1.9 compat: no Unicode normalization support yet" without any references. Does anyone know whether this is still true for ruby 1.9 and for which minor versions of 1.9?
Ruby doesn't support unicode normalization natively, but you can use this gem:
https://github.com/knu/ruby-unf
Hope that helps.
So using Iconv for transliteration is not working properly in 1.9 or?
You can still use it, but AFAIK iconv support is going to be dropped. The encoding system in 1.9 will let you convert between encodings just fine, but it doesn't do transliteration.
Probably the best path would either be to extract the iconv extension from Ruby and provide it as a gem, or to use an existing gem like this one:
https://github.com/rsl/stringex
I know it's not the ideal answer, but I hope it helps.
iconv's //TRANSLIT is not Unicode normalization.
Moreover iconv translit works only with glibc and GNU libiconv, and the actual behavior is not documented.
Ah. Thanks for the clarification!
