UPDATE 3
Well, it turns out that im stupid. Tried putting "sv_SE" into all sorts of files. Also tried this in config.php after the suggestion in this thread. Tho' I never tried without .utf8 -utf8.
$config['locale'] = 'sv_SE';
UPDATE 2
Upgraded => 1.8.1
UPDATE
This problem seem to only occour when using Internet Explorer
This might be an odd question. I just noticed that, when I go to adminlogin first time language markup is English. If I login and log off, the language is now Swedish (the one I specified to use).
Look at this,
This is when I first access the adminpage. And its all in english.

This is after I logged in, and then logout. And its all swedish.

NOTE, I checked my older customer install of CMSMS (1.7.x) and there is no problem with this.
So, as you might know. Im running 1.8 (Madagascar) on this one.
Thanks,
Br John