UFT8 String looks great in MySql, garbage in browser

rubyonrails-talk
1

Hi,

I have some seed data in MySql which contains non-ASCII characters (as you'd imagine as these are worldwide placenames that I'm storing). In MySql on the command line they look great, they're all formatted perfectly, including Arabic and Chinese characters and they look great.

However, when I ask my Rails model for one of these fields, and render that in the browser, all the lovely work is undone, and I get garbage character spew.

E.g.:

In the commandline, mySQL: "Чанчунь,长春" (Good!) In the browser: "K’ua-ch’eng-tzu,Чанчунь,长春" (Bad!!)

So clearly this is an encoding issue, here's my setup:

Table created via migration with ":options => "DEFAULT CHARSET=utf8"" Database.yml contains "encoding: utf8" HTML is served with the meta tag containing "charset=utf-8"

If I puts directly to the command line in the controller, I get the same (bad) spew, so I don't think it's the browser screwing up. All I'm doing to fetch the string is something like "@name = City.first.name"

Any ideas?

Thanks!

-Nex

2

What encoding is being set in the HTTP headers, if any?

What is the exact meta tag you are using?

3

Michael Graff wrote:

What encoding is being set in the HTTP headers, if any?

What is the exact meta tag you are using?

The meta tag is:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>

I'm not sure how to inspect the HTTP headers as they're returned from rails, without breaking out tcpdump what's the best way to see the header Rails is creating?

Thanks.

- Peter

4

continued...

I try printing out the response object just before my controller falls through to the view, and the header just has one value set:

{"Cache-Control"=>"no-cache"}

5

The Firebug plugin to Firefox works pretty well. Or just use telnet from the command line.

6

Hassan Schroeder wrote:

7

Peter Laurens wrote: [...]

Firebug reports the response header content-type as:

"Content-Type text/html; charset=utf-8"

So your HTTP headers are OK and the database is in the right encoding. Now for my stupid question: have you got the text encoding in the browser set to something other than auto-detect or UTF-8? Best,

8

curl -I http://…/

is my favorite. Only prints back the headers the server sends and not the body...

9

Firebug reports the response header content-type as:

"Content-Type text/html; charset=utf-8"

Excellent :slight_smile:

Just to cover all the DB bases -- see if *everything* is using utf-8, e.g.

10

Yep, exactly, set them in your /etc/my.cnf and relaunch mysqld.

11

Thanks a lot to Hassan, I have finally sorted this problem out properly. There were a few more steps:

1. I had to make sure that the right values for utf8 were set in my.cnf. Looking for the right set of values I found the following:

[mysqld] init_connect=’SET collation_connection = utf8_general_ci’ init_connect=’SET NAMES utf8′ default-character-set=utf8 character-set-server=utf8 collation-server=utf8_general_ci skip-character-set-client-handshake

2. I learned that once created, each database, table, and column has a charset assigned. And these were not automatically changed with the change to my.cnf. To complete the fix, I would either have to do:

alter database db_name charset=utf8; alter table t_name charset=utf8; AND a similar alter for the columns.

Instead I just dropped everything and recreated the databases (which then picked up the new charset) and it worked, finally. Wow, that was an interesting bit of setup!

Thanks to all for the help.