CVS from Excel Encoding issue

rubyonrails-talk
1

Sorry for the cross post but no takers in the Ruby forum:

Using Ruby 1.8 (for Rails 2.3.5)

I have a cvs file that was exported from an Excel file.

Some of the text contains characters like “Test in quotes” (I think these are smart quotes)

I can properly change this to UTF-8 via:

Iconv.conv("utf-8", "cp1251", text)

Other text contains text like spanish characters like Señora. I can properly change this to UTF-8 like this:

Iconv.conv("utf-8", "iso-8859-1", text)

However I cannot figure out how to get some text that has both types of characters like this:

“Test in quotes” Señora

If I don't get these properly into utf-8 they do not persist properly in mysql (they appear as ?)

Any ideas? Any help is greatly appreciated.

Thanks in advance.

2

Bump - anyone?

3

You may want to try converting from Windows-1252 (not 1251), as the "smart quotes" are the same codepoints in both, but 1252 has the Spanish characters rather than the Cyrillic characters found in 1251.

--Matt Jones

4

Matt Jones wrote: