Character encoding issues - Mageia 2

[dedanna1029]

Regarding this post, it appears I'm having character encoding issues all around. Every file manager (that I can start, that is; nautilus has issues right now), there are black question marks, etc. in place of foreign characters in folder and filenames, and those files/folders are not recognized, even though they are indeed "there" ("does not exist" error). In every distro as of late, font encoding was something I could take for granted; they all worked, whatever locale I was using (U.S., UK, whatever). I'm currently using the UK one. Is there something up possibly with UTF encoding, or ?

Thanks.

[viking60]

Do you have a screenshot?

[dedanna1029]

Sure. PCMANFM, which just shows the X in the box when it doesn't recognize something:



In dolphin, notice my former piano teacher, Susan Svrcek, on the right, then when I try to open her folder... (She just released another cd, btw! ):



These are just examples of hundreds, as I listen to a lot of symphonic rock, and a lot of it is foreign. Finland, etc.

I've just installed every font and font pack, and locale available in Mageia 2, no luck.

Edit: Keep in mind, that my music library is currently on a Mageia 2 server, but that shoudln't make any diff - every machine here is on up-to-date Mageia 2. Thanks.

[dedanna1029]

More examples: Chanee and N'Evergreen, Royskopff, more Sigur Ros, etc. etc. ... (not typed in this post with the correct characters, sorry):



Sorry for the blip on the second one here; you can still see though, the black ? in Chanee and N'Evergreen.

HOWEVER, in all of this, I can reach these folders in terminal:

Code: Select all

[dedanna@dedanna ~]$ cd /media/KimMusic/Sigur\ R\ufffds\ -\ Discography/
[dedanna@dedanna Sigur R\ufffds - Discography]$ ls
1997 - Von/            2002 - ( )/                 2007 - Hvarf-Heim/
1999 - ?g?tis Byrjun/  2003 - Hlemmur Soundtrack/  2008 - Me? su? ? eyrum vi? spilum endalaust/
2001 - Rimur (EP)/     2005 - Takk.../             2012 - Valtari/
[dedanna@dedanna Sigur R\ufffds - Discography]$ cd 1999\ -\ \ufffdg\ufffdtis\ Byrjun/
[dedanna@dedanna 1999 - \ufffdg\ufffdtis Byrjun]$ ls
01 - Intro.mp3           04 - Flugufrelsarinn.mp3                  07 - Vi?rar vel til loft?rasa.mp3  10 - Avalon.mp3
02 - Svefn-g-englar.mp3  05 - N? batter?.mp3                       08 - Olsen Olsen.mp3
03 - Star?lfur.mp3       06 - Hjarta? hamast (bamm bamm bamm).mp3  09 - ?g?tis byrjun.mp3
[dedanna@dedanna 1999 - \Uffffffffg\Uffffffff

EXCEPT, I can't copy/paste it right into here. Note the Uffffffffg\Uffffff stuff.... that's how these things copied from termnal, but they didn't display that way in the terminal. MAN, this SUCKS!



Edited to correct code tag - I'm buzzed off Wyld Wood Cider, what can I say?

[viking60]

Hold your horses - one tiny thing at the time please
Røyksopp has the letter ø from Valhalla. If you copy ø or Ø into a folder name you will get the same error so it looks like you are not using UTF 8 -unicode (I think - it is uni-something) which should present it nicely. Inferior cultures do not have the letter ø (and Røyksopp has it in the name because they are a Norwegian group).
So yes it seems to be a UTF8 thing. (I think you need to adjust some global setting in MCC)
I am going to read the rest of your post now.....

[dedanna1029]

OK. The global setting being... ?

FYI:

Code: Select all

[root@dedanna ~]# urpmq -i unicode -a
    $MIRRORLIST: media/debug/core/updates/media_info/20130318-214021-info.xml.lzma
Name        : rxvt-unicode                                                                                                 
Version     : 9.12
Release     : 2.1.mga2
Group       : Terminals
Size        : 2694997                      Architecture: i586
Source RPM  : rxvt-unicode-9.12-2.1.mga2.src.rpm
URL         : http://dist.schmorp.de/rxvt-unicode
Summary     : A color VT102 terminal emulator for the X Window System
Description :
Rxvt-unicode is a clone of the well known terminal emulator rxvt, modified to
store text in Unicode (either UCS-2 or UCS-4) and to use locale-correct input
and output. It also supports mixing multiple fonts at the same time, including
Xft fonts.

Name        : rxvt-unicode-debug
Version     : 9.12
Release     : 2.1.mga2
Group       : Development/Debug
Size        : 8201329                      Architecture: i586
Source RPM  : rxvt-unicode-9.12-2.1.mga2.src.rpm
URL         : http://dist.schmorp.de/rxvt-unicode
Summary     : Debug information for package rxvt-unicode
Description :
This package provides debug information for package rxvt-unicode.
Debug information is useful when developing applications that use this
package or when debugging this package.

[root@dedanna ~]# urpmq utf-8
No package named utf-8

[root@dedanna ~]# urpmq utf8
mutt-utf8

... and I couldn't give two craps about mutt...

[viking60]

No urxvt is a terminal and Mutt is wrong too (If you want to find a program it is locale). You will not find it like that. You need to set it in your locales. I would have thought that you could do it in MCC somewhere
For gnome-terminal and Konsole you can set it via the menus (LANG=en_US.UTF-8)
Here is some info
Simply setting your locale to utf-8 should be enough
Maybe you have a font that cannot display ø?

[dedanna1029]

There's others too. The i with l'accent grave; the a with the two dots on top; those are just a couple of examples. I have the Finnish, Norse, and several other locales installed, but I'll try the LANG=en_UK or US with utf8 (if it gives me utf8 option (I'm on UK locale right now; do you think it best to go US while I'm here in the UK? I have it installed).

FYI: Our two Mageia servers are also having this problem.

[jkerr82508]

I have created files with foreign characters in their names, including Røyksopp. Dolphin and konsole display them correctly.

Kcharselect can display a whole range of "foreign" characters, any of which it seems can be used by any KDE program.

I have not installed any additional locales. In /etc/locale I have only the series of en_ variants that were installed by default.

Mageia uses UTF-8 by default - unless you change it in MCC - System - Manage localisation - Choose language (Advanced).

Jim

[dedanna1029]

non-UTF-8 encoding is not ticked there in Advanced.

I'm set to English.

I see characters in both character choosers in KDE and Gnome, but not all.

I even see the characters that should be used for these folders and files.

The point is, it's not working. They are displayed in file manager without their correct characters, they won't let me rename them in file managers, because file managers think the files/folders aren't there because of the foreign characters.

I also can not copy/paste foreign characters correctly from the respective clipboards (Klipper and Parcellite), as I have shown. If I copy straight out of charset, for instance, it is the correct character I'm copying. When I paste it, it comes out as the above.

We also have this issue on two other Mageia 2 machines.

[dedanna1029]

I have just found a bit of a helper here, but it's not a resolution, because the option is not always there to use.

If I just go "Paste" when I paste in foreign characters, it will do as I said above. However, if I right-click and choose "Paste as plain text", it will paste correctly. I don't know if this has to do with the fact that I'm now using Iron browser (keep in mind I haven't tried this in Dolphin yet), and not FF for the web, but that option is a help when it's available:

Code: Select all

Right-clicked and "Paste as plain text":
Cela m'aiderait à rafraîchir mon français

[dedanna1029]

Yep, I've borked it. This just clicked in full, the second part:

Mageia uses UTF-8 by default - unless you change it in MCC - System - Manage localisation - Choose language (Advanced).

I had to install several locales, because my mum and I are working on several different languages, and I needed to be able to use them, and be able to change them at random times using the input method software in Gnome (System Settings--->Region and Language-->Set up all of the ones you want). I am also in the UK, so had set country to UK in MCC deliberately, with Old UTF-8 encoding option ticked, because I'm currently in the UK; had plans to change it back to US when I go back there. None of that, I don't think, is relevant except for the fact that I did change it in MCC to "English", and country to UK. By doing this, I changed the default.

I don't see any way to resolve this, other than a reinstall of /, and just accept the default. In MCC, there is no option just to change to UTF-8; there is only option to change the language to a specific language, which I've tried to no avail (back to English [United States], using the Old UTF-8 encoding option there).

My question before I do this, is, does anyone see any other way out other than formatting / and reinstalling?

Thanks.