adding i18n support to Rails' transliterator

Hi all,

The other day I sent a patch to Rails to improve
ActiveSupport::Inflector's support for transliterating UTF-8 Latin
characters to ASCII.

In the discussion of the patch, I mentioned the idea of making
Inflector.transliterate use Rails' i18n facilities, and Jeremy Kemper
and Yaroslav Markin encouraged me to pursue the idea.

http://github.com/rails/rails/commit/dceef0828a23e8298dd9a9aab1a33c49e84f17d6#activesupport/lib/active_support/inflector/transliterate.rb-P50

I've been working on support for this and would like to ask for some
feedback before sending a patch to Rails. My work can be found at:

http://github.com/norman/rails/tree/translit

Basically the new code lets you do things like this:

    I18n.backend.store_translations(:de, :support =>
{:transliterations => {"ü" => "ue", "ö" => "oe"}})
    assert_equal "Juergen Koehler",
ActiveSupport::Inflector.transliterate("Jürgen Köhler")

Yaroslav, if you look in the tests you'll see I also basically stole
your Russian transliterator verbatim. :slight_smile:

One of my main concerns has been keeping reasonably good performance
in a thread-safe manner. The new code memoizes the available
transliterators in a class variable, which I'm a little hesitant about
doing, but I think will be ok. If someone more experienced with
thread-safety issues could advise me on that I'd be grateful.

Regards,

Norman

Hey. Really like your patch :slight_smile:

Quick feedback:

— we use i18n.* namespace for stuff like pluralization — maybe this is a better fit than support.*?..

— I’m not really sure about obj.respond_to?(:transliterate) — maybe we should do it pluralize-like and support lambdas as argument? As well as a Hash, of course. It’s just that storing a lambda .rb translation is a bit easier than storing classes :slight_smile: Also, I18n 0.3 has quite good lambda localization support already.

Not really sure about performance issues and solving them this way, I believe stuff like this is handled by pluggable back-ends; I hope Sven or anyone else can correct me on that.

Hey. Really like your patch :slight_smile:

Thanks a lot for the feedback.

Quick feedback:
— we use i18n.* namespace for stuff like pluralization — maybe this is a
better fit than support.*?..

Ok, I'll change that.

— I'm not really sure about obj.respond_to?(:transliterate) — maybe we
should do it pluralize-like and support lambdas as argument? As well as a
Hash, of course. It's just that storing a lambda .rb translation is a bit
easier than storing classes :slight_smile: Also, I18n 0.3 has quite good lambda
localization support already.

That's actually what I wanted to do at first, but I didn't see how I
could access the lambda without calling it:

http://gist.github.com/366393

Am I missing something?

Not really sure about performance issues and solving them this way, I
believe stuff like this is handled by pluggable back-ends; I hope Sven or
anyone else can correct me on that.

Ok, I'll take a look into the pluggable backends.

Sorry to answer my own post, I realized I can simply pass :resolve =>
false to get the Proc directly. Now I'm looking more closely at i18n's
pluralizations and will see about implementing transliterations in a
similar manner.

Hey,

this looks awesome :slight_smile:

I've always wanted something like this in the I18n gem but never came around to it.

Quick feedback:
— we use i18n.* namespace for stuff like pluralization — maybe this is a
better fit than support.*?..

Ok, I'll change that.

I agree.

That's actually what I wanted to do at first, but I didn't see how I
could access the lambda without calling it:

http://gist.github.com/366393

Am I missing something?

Yeah, that's a feature :slight_smile: I guess we need to resolve that somehow. Can you contact me offlist about this?

Not really sure about performance issues and solving them this way, I
believe stuff like this is handled by pluggable back-ends; I hope Sven or
anyone else can correct me on that.

Ok, I'll take a look into the pluggable backends.

Why not just add a .transliterate method to the I18n gem api? Seems like the right level of abstraction to me. The implementation could then easily be exchanged/extended just as other features in I18n.