mysql2 and swedish characters

When using the mysql2 gem with my excising databases all non-standard
chars gets messed up:

från is shown as frÃ¥n in the browser

Is this possible to fix or do I have to continue using the old mysql-
gem?

IIRC mysql2 forces use of utf8. If you're elsewhere telling the
browser that you're using a different character set then you'd get
unwanted results.

Fred

I think it's more a question of how the data is stored in the
database.
In my layout i have:
<meta http-equiv="content-type" content="text/html;charset=UTF-8" />
and in application.rb:
config.encoding = "utf-8"

:slight_smile: j

I think it's more a question of how the data is stored in the
database.
In my layout i have:
<meta http-equiv="content-type" content="text/html;charset=UTF-8" />
and in application.rb:
config.encoding = "utf-8"

Also possible. If previously the database connection was configured as
some latin1 variant and your columns were also latin1 but you were
stuffing utf8 data into them then switching to mysql2 would cause data
to appear messed up.

Fred

That seems to work, the problem is when reading the data using mysql2.
In the database å is stored as Ã¥.
When I write to the database using mysql2, without having changed it,
it writes å.

== Welcome to Vidli

Vidli ( http://vidli.com ) is the first of its kind Open Source Video eCommerce platform. Specifically designed for video content owners that are looking to setup a distribution storefront. Vidli is built on the Ruby on Rails v3 framework.

Out of the box, Vidli integrates with PayPal Express checkout for payment processing and Amazon S3 for video streaming and storage.

== Getting Started

Download the source code

   git clone http://github.com/vidli/vidli

Copy the database.yml.example file to database.yml. Update with your login and password to your MySQL database.

   cp config/database.yml.example config/database.yml

Configure your AWS settings

   cp config/amazon_s3.yml.example config/amazon_s3.yml

Create the development and production buckets on AWS via http://console.aws.amazon.com that matches your config file.

Upload the lib/crossdomain.xml file to each bucket. Make sure permissions have Everyone -> Open/Download.

Signup at PayPal for a developer account

   http://developer.paypal.com

Setup your Vidli app config and ActiveMerchant for PayPal payments

   cp config/vidli_config.yml.example config/vidli_config.yml

== Try it out

Visit http://demo.vidli.com to try out the Vidli app for yourself.

== Who is Vidli for?

- Video content creators
- Video distributors

== Vidli comes with

- Simple, easy to use video content management system
- Non-commercial use JW Player video player (when you go live this will need to be purchased)
- Seamless integration with PayPal Express
- Integration with Amazon S3 for video storage and playback

== License

Vidli is released under the MIT license.

That seems to work, the problem is when reading the data using mysql2.
In the database å is stored as å.
When I write to the database using mysql2, without having changed it,
it writes å.

What does mysql think the column types are ? Before using mysql2, what
encoding was set in your database.yml ?

Fred

No encoding set in database.yml, cp1252 set in mysql. By it self I
guess.

No encoding set in database.yml, cp1252 set in mysql. By it self I
guess.

that sounds like you should change the encoding for your existing
tables/columns to utf8

One way of doing this is to alter the columns to blobs and then back
to a text column with the correct encoding (as documented at the
bottom of http://dev.mysql.com/doc/refman/5.0/en/alter-table.html)

Fred

Yes, that seems to work, but takes some time.
Thanks!

Bummer, this didn’t work for me. I think I’m having a similar problem.

I was using mysql gem, database was utf8, but I was getting an occasional template encoding error. That prompted me to update my driver to mysql2 (which solved my template problem), but resulted in formerly “correct” characters getting botched.

Björn became Björn. Or maybe it always was that, but mysql driver was somehow presenting it as we want to see it. Regardless, going to mysql2 left me with a lot of screwed up names :frowning:

I also tried the column -> blob -> text trick, and sadly that didn’t appear to change anything.

Do you think there’s some simple algorithm I can use to map screwy things to their correct versions?

Thanks!

More info…

In rails console, mysql driver shows this for the column value: Bj\xF6rn
mysql2 driver shows this for the column value: Björn

F6 hex is the UTF8 code for o umlaut, so the mysql(1) view seems more correct than the mysql2 view of the column value…

Bummer, this didn't work for me. I think I'm having a similar problem.

I was using mysql gem, database was utf8, but I was getting an occasional
template encoding error. That prompted me to update my driver to mysql2
(which solved my template problem), but resulted in formerly "correct"
characters getting botched.

Björn became Björn. Or maybe it always was that, but mysql driver was
somehow presenting it as we want to see it. Regardless, going to mysql2
left me with a lot of screwed up names :frowning:

I also tried the column -> blob -> text trick, and sadly that didn't appear
to change anything.

I once had a case where through an accident of history, some data was
doubly bad, so I had to go through the text->blob->text approach
multiple times

F6 is actually a latin1 code point, which suggests that you still
haven't got things in utf8 properly

Fred

Ahh, I’ll make a few more blob/text round trips and see if that helps.

Yeah, the F6 thing was created by mysql(1) driver. Now mysql2 apparently doesn’t understand.

Thanks!