You can now already use any Unicode characters in file names. No kernel or file utilities need modifications. This is because file names in the kernel can be anything not containing a null byte, and '/' is used to delimit subdirectories. When encoded using UTF-8, non-ASCII characters will never be encoded using null bytes or slashes. All that happens is that file and directory names occupy more bytes than they contain characters. For example, a filename consisting of five greek characters will appear to the kernel as a 10-byte filename. The kernel does not know (and does not need to know) that these bytes are displayed as greek.
This is the general theory, as long as your files stay inside Linux. On filesystems which are used from other operating systems, you have mount options to control conversion of filenames to/from UTF-8:
Ttys are some kind of bidirectional pipes between two program, allowing fancy features like echoing or command-line editing. When in an xterm, you execute the "cat" command without arguments, you can enter and edit any number of lines, and they will be echoed back line by line. The kernel's editing actions are not correct, especially the Backspace (erase) key and the tab bey are not treated correctly.
To fix this, you need to:
You will need a program to convert your locally (probably ISO-8859-1) encoded texts to UTF-8. (The alternative would be to keep using texts in different encodings on the same machine; this is not fun in the long run.) One such program is `iconv', which comes with glibc-2.1. Simply use
$ iconv --from-code=ISO-8859-1 --to-code=UTF-8 < old_file > new_file
Here are two handy shell scripts, called "i2u" i2u.sh (for ISO to UTF conversion) and "u2i" u2i.sh (for UTF to ISO conversion). Adapt according to your current 8-bit character set.
If you don't have glibc-2.1 and iconv installed, you can use GNU recode 3.5 instead. "i2u" i2u_recode.sh is "recode ISO-8859-1..UTF-8", and "u2i" u2i_recode.sh is "recode UTF-8..ISO-8859-1". ftp://ftp.iro.umontreal.ca/pub/recode/recode-3.5.tar.gz ftp://ftp.gnu.org/pub/gnu/recode/recode-3.5.tar.gz Notes: You need GNU recode 3.5 or newer. To compile GNU recode 3.5 on platforms without glibc2 (i.e. on all platforms except recent Linux systems), you need to configure it with --disable-nls, otherwise it won't link. Newer development versions of GNU recode with CJK support are available at http://www.iro.umontreal.ca/contrib/recode/.
Or you can also use CLISP instead. Here are "i2u" i2u.lisp and "u2i" u2i.lisp written in Lisp. Note: You need a CLISP version from July 1999 or newer. ftp://clisp.cons.org/pub/lisp/clisp/source/clispsrc.tar.gz.
Other data conversion programs, less powerful than GNU recode, are `trans' ftp://ftp.informatik.uni-erlangen.de/pub/doc/ISO/charsets/trans113.tar.gz, `tcs' from the Plan9 operating system ftp://ftp.informatik.uni-erlangen.de/pub/doc/ISO/charsets/tcs.tar.gz, and `utrans'/`uhtrans'/`hutrans' ftp://ftp.cdrom.com/pub/FreeBSD/distfiles/i18ntools-1.0.tar.gz by G. Adam Stanislav <adam@whizkidtech.net>.
For the repeated conversion of files to UTF-8 from different character sets, a semi-automatic tool can be used: to-utf8 presents the non-ASCII parts of a file to the user, lets him decide about the file's original character set, and then converts the file to UTF-8.
You may have the following environment variables set, containing locale names:
override for LC_MESSAGES, used by GNU gettext only
override for all other LC_* variables
individual variables for: character types and encoding, natural language messages, sorting rules, number formatting, money amount formatting, date and time display
default value for all LC_* variables
man 7 locale' for a detailed description.)
Each of the LC_* and LANG variables can contain a locale name of the following form:
language[_territory[.codeset]][@modifier]
where language is an ISO 639 language code (lower case), territory is an ISO 3166 country code (upper case), codeset denotes a character set, and modifier stands for other particular attributes (for example indicating a particular language dialect, or a nonstandard orthography).
LANGUAGE can contain several locale names, separated by colons.
In order to tell your system and all applications that you are using UTF-8, you need to add a codeset suffix of UTF-8 to your locale names. For example, if you were using
LC_CTYPE=de_DE
LC_CTYPE=de_DE.UTF-8
You do not need to change your LANGUAGE environment variable. GNU gettext has the ability to convert translations to the right encoding. Until glibc-2.2 is released, all you have to do is to set the OUTPUT_CHARSET environment variable.
$ export OUTPUT_CHARSET=UTF-8
If you have glibc-2.1 or glibc-2.1.1 or glibc-2.1.2 installed, first check using "localedef --help" that the system directory for character maps is /usr/share/i18n/charmaps. Then apply to the file /usr/share/i18n/charmaps/UTF8 the patch glibc21.diff or glibc211.diff or glibc212.diff, respectively. Then create the support files for each UTF-8 locale you intend to use, for example:
$ localedef -v -c -i de_DE -f UTF8 /usr/share/locale/de_DE.UTF-8
You typically don't need to create locales named "de" or "fr" without country suffix, because these locales are normally only used by the LANGUAGE variable and not by the LC_* variables, and LANGUAGE is only used as an override for LC_MESSAGES.
The glibc-2.2 will support multibyte locales, in particular the UTF-8 locales created above. But glibc-2.1.x does not really support it. Therefore the only real effect of the above creation of the /usr/share/locale/de_DE.UTF-8/* files is that `setlocale(LC_ALL,"")' will return "de_DE.UTF-8", according to your environment variables, instead of stripping off the ".UTF-8" suffix.
To add support for the UTF-8 locale, you should build and install the following three libraries:
$ export LD_PRELOAD=/usr/local/lib/libutf8_plug.so:/usr/local/lib/libiconv_plug.so:/usr/local/lib/libintl_plug.so
This entire thing will not be necessary any more once glibc-2.2 comes out.