CMS Made Simple Forums

Ok, I have the age-old problem of UTF-8 content stored in Latin1-collated mysql tables. I am trying to convert my tables to UTF-8.

I have got fairly far converting the data. I changed the table's contents to binary, then changed to text in UTF-8. The site responded well, content is UTF-8 and characters display properly.

However, in the admin panel, UTF-8 content is not displaying properly. Either a hollow square (IE8) or a question mark (Firefox) when looking at the content in TinyMCE, both with WYSIWYG on and off. Both browsers are set to display in UTF-8 when in the admin panel.

So, why would the UTF-8 characters display correctly on the site, but not in the Admin panel?

Thanks.

oliver341 wrote: I have got fairly far converting the data. I changed the table's contents to binary, then changed to text in UTF-8. The site responded well, content is UTF-8 and characters display properly.

not always is good switch encoding in a SQL server ....
Sometime is necessary an update of content

Alby

alby wrote: not always is good switch encoding in a SQL server ....
Sometime is necessary an update of content

Yeah, but the site itself looks fine. It's just the admin panel which is displaying the UTF-8 characters wrongly. I can't work out why.

oliver341 wrote: Yeah, but the site itself looks fine. It's just the admin panel which is displaying the UTF-8 characters wrongly. I can't work out why.

Your "set_names" setting in config.php and your DB server?

Alby

alby wrote: Your "set_names" setting in config.php and your DB server?

In my config I have:
$config['set_names'] = false;

Is that wrong?

I have looked in phpMyAdmin and it says:
MySQL charset: UTF-8 Unicode (utf8)

I think the database is fully functioning in utf-8.

oliver341 wrote:

alby wrote: Your "set_names" setting in config.php and your DB server?

In my config I have:
$config['set_names'] = false;

It must true for "real" utf8

oliver341 wrote: MySQL charset: UTF-8 Unicode (utf8)

You don't say mysql version

Alby

alby wrote: It must true for "real" utf8

OK, I'll have a play with this setting.

You don't say mysql version

Server version: 5.0.82-community

Just an update on this.

After messing around with mysql character enconding, I realised, after flushing the site cache, that many pages had become truncated. Of course I make regular backups of the mysql database, so I restored a backup. I had fully expected this would introduce weird characters into the website as the mysql character encoding was set to latin1, but the content was utf-8.

Thankfully, to my surprise, despite the character encoding mismatch, the site is not displaying any characters incorrectly. I think I will now back out of my attempts to convert the character encoding!

I note that fresh installations configure mysql character enconding to utf-8. I must have installed a CMSMS version which still set mysql enconding to latin1.

oliver341 wrote: .. that many pages had become truncated.

It's normal, many utf-8 chars have necessity of more store space

oliver341 wrote: I note that fresh installations configure mysql character enconding to utf-8. I must have installed a CMSMS version which still set mysql enconding to latin1.

You say for cmsms or mysql?

cmsms upgrade set $config['set_names'] to false, install to true
mysql depend of your server configuration

Alby

alby wrote:cmsms upgrade set $config['set_names'] to false, install to true
mysql depend of your server configuration

I installed CMSMS about two years ago and all my tables were (and are) set to latin1 encoding. However I've just installed the latest CMSMS on the same web host and all the tables are set to utf-8.