adding i18n support to Rails' transliterator

rubyonrails-core
1

Hi all,

The other day I sent a patch to Rails to improve ActiveSupport::Inflector's support for transliterating UTF-8 Latin characters to ASCII.

In the discussion of the patch, I mentioned the idea of making Inflector.transliterate use Rails' i18n facilities, and Jeremy Kemper and Yaroslav Markin encouraged me to pursue the idea.

http://github.com/rails/rails/commit/dceef0828a23e8298dd9a9aab1a33c49e84f17d6#activesupport/lib/active_support/inflector/transliterate.rb-P50

I've been working on support for this and would like to ask for some feedback before sending a patch to Rails. My work can be found at:

http://github.com/norman/rails/tree/translit

Basically the new code lets you do things like this:

    I18n.backend.store_translations(:de, :support => {:transliterations => {"ü" => "ue", "ö" => "oe"}})     assert_equal "Juergen Koehler", ActiveSupport::Inflector.transliterate("Jürgen Köhler")

Yaroslav, if you look in the tests you'll see I also basically stole your Russian transliterator verbatim. :slight_smile:

One of my main concerns has been keeping reasonably good performance in a thread-safe manner. The new code memoizes the available transliterators in a class variable, which I'm a little hesitant about doing, but I think will be ok. If someone more experienced with thread-safety issues could advise me on that I'd be grateful.

Regards,

Norman

2

Hey. Really like your patch :slight_smile:

Quick feedback:

— we use i18n.* namespace for stuff like pluralization — maybe this is a better fit than support.*?..

— I’m not really sure about obj.respond_to?(:transliterate) — maybe we should do it pluralize-like and support lambdas as argument? As well as a Hash, of course. It’s just that storing a lambda .rb translation is a bit easier than storing classes :slight_smile: Also, I18n 0.3 has quite good lambda localization support already.

Not really sure about performance issues and solving them this way, I believe stuff like this is handled by pluggable back-ends; I hope Sven or anyone else can correct me on that.

3

Hey. Really like your patch :slight_smile:

Thanks a lot for the feedback.

Quick feedback: — we use i18n.* namespace for stuff like pluralization — maybe this is a better fit than support.*?..

Ok, I'll change that.

— I'm not really sure about obj.respond_to?(:transliterate) — maybe we should do it pluralize-like and support lambdas as argument? As well as a Hash, of course. It's just that storing a lambda .rb translation is a bit easier than storing classes :slight_smile: Also, I18n 0.3 has quite good lambda localization support already.

That's actually what I wanted to do at first, but I didn't see how I could access the lambda without calling it:

Am I missing something?

Not really sure about performance issues and solving them this way, I believe stuff like this is handled by pluggable back-ends; I hope Sven or anyone else can correct me on that.

Ok, I'll take a look into the pluggable backends.

4

Sorry to answer my own post, I realized I can simply pass :resolve => false to get the Proc directly. Now I'm looking more closely at i18n's pluralizations and will see about implementing transliterations in a similar manner.

5

Hey,

this looks awesome :slight_smile:

I've always wanted something like this in the I18n gem but never came around to it.

Quick feedback: — we use i18n.* namespace for stuff like pluralization — maybe this is a better fit than support.*?..

Ok, I'll change that.

I agree.

That's actually what I wanted to do at first, but I didn't see how I could access the lambda without calling it:

gist:366393 · GitHub

Am I missing something?

Yeah, that's a feature :slight_smile: I guess we need to resolve that somehow. Can you contact me offlist about this?

Not really sure about performance issues and solving them this way, I believe stuff like this is handled by pluggable back-ends; I hope Sven or anyone else can correct me on that.

Ok, I'll take a look into the pluggable backends.

Why not just add a .transliterate method to the I18n gem api? Seems like the right level of abstraction to me. The implementation could then easily be exchanged/extended just as other features in I18n.